Efficient estimation and model selection in large graphical models

We develop a computationally efficient method to determine the interaction structure in a multidimensional binary sample. We use an interaction model based on orthogonal functions, and give a result on independence properties in this model. Using this result we develop an efficient approximation algorithm for estimating the parameters in a given undirected model. To find the best model, we use a heuristic search algorithm in which the structure is determined incrementally. We also give an algorithm for reconstructing the causal directions, if such exist. We demonstrate that together these algorithms are capable of discovering almost all of the true structure for a problem with 121 variables, including many of the directions.     

Convergence behavior of an iterative reweighting algorithm to compute multivariate M-estimates for location and scatter

The iteratively reweighting algorithm is one of the widely used algorithm to compute the M-estimates for the location and scatter parameters of a multivariate dataset. If the M estimating equations are the maximum likelihood estimating equations from some scale mixture of normal distributions (e.g. from a multivariate t-distribution), the iteratively reweighting algorithm is identified as an EM algorithm and the convergence behavior of which is well established. However, as Tyler (J. Roy. Statist. Soc. Ser. B 59 (1997) 550) pointed out, little is known about the theoretical convergence properties of the iteratively reweighting algorithms if it cannot be identified as an EM algorithm. In this paper, we consider the convergence behavior of the iteratively reweighting algorithm induced from the M estimating equations which cannot be identified as an EM algorithm. We give some general results on the convergence properties and, we show that convergence behavior of a general iteratively reweighting algorithm induced from the M estimating equations is similar to the convergence behavior of an EM algorithm even if it cannot be identified as an EM algorithm.     

Simulation of right and left truncated gamma distributions by mixtures

We study the properties of truncated gamma distributions and we derive simulation algorithms which dominate the standard algorithms for these distributions. For the right truncated gamma distribution, an optimal accept–reject algorithm is based on the fact that its density can be expressed as an infinite mixture of beta distribution. For integer values of the parameters, the density of the left truncated distributions can be rewritten as a mixture which can be easily generated. We give an optimal accept–reject algorithm for the other values of the parameter. We compare the efficiency of our algorithm with the previous method and show the improvement in terms of minimum acceptance probability. The algorithm proposed here has an acceptance probability which is superior to e/4.     

ON LATOUCHE–RAMASWAMI'S LOGARITHMIC REDUCTION ALGORITHM FOR QUASI-BIRTH-AND-DEATH PROCESSES

We consider Latouche–Ramaswami's logarithmic reduction algorithm for solving quasi-birth-and-death models. We shall present some theoretical properties concerning convergence of the algorithm and discuss numerical issues arising in finite precision implementations. In particular, we shall present a numerically more stable implementation. A rounding error analysis together with numerical examples are given to demonstrate the higher accuracy achieved by the refined implementation.     

Variable selection in quantile regression when the models have autoregressive errors

This paper considers a problem of variable selection in quantile regression with autoregressive errors. Recently, Wu and Liu (2009) investigated the oracle properties of the SCAD and adaptive-LASSO penalized quantile regressions under non identical but independent error assumption. We further relax the error assumptions so that the regression model can hold autoregressive errors, and then investigate theoretical properties for our proposed penalized quantile estimators under the relaxed assumption. Optimizing the objective function is often challenging because both quantile loss and penalty functions may be non-differentiable and/or non-concave. We adopt the concept of pseudo data by Oh et al. (2007) to implement a practical algorithm for the quantile estimate. In addition, we discuss the convergence property of the proposed algorithm. The performance of the proposed method is compared with those of the majorization-minimization algorithm (Hunter and Li, 2005) and the difference convex algorithm (Wu and Liu, 2009) through numerical and real examples.     

Bivariate Kumaraswamy distribution: properties and a new method to generate bivariate classes

In this paper, we introduce a bivariate Kumaraswamy (BVK) distribution whose marginals are Kumaraswamy distributions. The cumulative distribution function of this bivariate model has absolutely continuous and singular parts. Representations for the cumulative and density functions are presented and properties such as marginal and conditional distributions, product moments and conditional moments are obtained. We show that the BVK model can be obtained from the Marshall and Olkin survival copula and obtain a tail dependence measure. The estimation of the parameters by maximum likelihood is discussed and the Fisher information matrix is determined. We propose an EM algorithm to estimate the parameters. Some simulations are presented to verify the performance of the direct maximum-likelihood estimation and the proposed EM algorithm. We also present a method to generate bivariate distributions from our proposed BVK distribution. Furthermore, we introduce a BVK distribution which has only an absolutely continuous part and discuss some of its properties. Finally, a real data set is analysed for illustrative purposes.     

Recursive distribution estimator defined by stochastic approximation method using Bernstein polynomials

We propose a recursive distribution estimator using Robbins-Monro's algorithm and Bernstein polynomials. We study the properties of the recursive estimator, as a competitor of Vitale's distribution estimator. We show that, with optimal parameters, our proposal dominates Vitale's estimator in terms of the mean integrated squared error. Finally, we confirm theoretical result throught a simulation study.     

Estimation and testing for semiparametric mixtures of partially linear models

In this paper, we study the estimation and inference for a class of semiparametric mixtures of partially linear models. We prove that the proposed models are identifiable under mild conditions, and then give a PL–EM algorithm estimation procedure based on profile likelihood. The asymptotic properties for the resulting estimators and the ascent property of the PL–EM algorithm are investigated. Furthermore, we develop a test statistic for testing whether the non parametric component has a linear structure. Monte Carlo simulations and a real data application highlight the interest of the proposed procedures.     

Using adaptively weighted large margin classifiers for robust sufficient dimension reduction

In this paper, we combine adaptively weighted large margin classifiers with Support Vector Machine (SVM)-based dimension reduction methods to create dimension reduction methods robust to the presence of extreme outliers. We discuss estimation and asymptotic properties of the algorithm. The good performance of the new algorithm is demonstrated through simulations and real data analysis.     

Non linear parametric mode regression

In this article, we propose a semi-parametric mode regression for a non linear model. We use an expectation-maximization algorithm in order to estimate the regression coefficients of modal non linear regression. We also establish asymptotic properties for the proposed estimator under assumptions of the error density. We investigate the performance through a simulation study.     

Approximate wavelet-based simulation of long memory processes

In this article, we investigate an algorithm for the fast O(N) and approximate simulation of long memory (LM) processes of length N using the discrete wavelet transform. The algorithm generates stationary processes and is based on the notion that we can improve standard wavelet-based simulation schemes by noting that the decorrelation property of wavelet transforms is not perfect for certain LM process. The method involves the simulation of circular autoregressive process of order one. We demonstrate some of the statistical properties of the processes generated, with some focus on four commonly used LM processes. We compare this simulation method with the white noise wavelet simulation scheme of Percival and Walden [Percival, D. and Walden, A., 2000, Wavelet Methods for Time Series Analysis (Cambridge: Cambridge University Press).].     

Distribution of busy period in stochastic fluid models

We consider the busy period in a stochastic fluid flow model with infinite buffer where the input and output rates are controlled by a finite homogeneous Markov process. We derive an explicit expression for the distribution of the busy period and we obtain an algorithm to compute it which exhibits nice numerical properties.

     

Likelihood inference for exchangeable continuous data with covariates and varying cluster sizes; use of the Farlie–Gumbel–Morgenstern model

This article investigates the Farlie–Gumbel–Morgenstern class of models for exchangeable continuous data. We show how the model specification can account for both individual and cluster level covariates, we derive insights from comparisons with the multivariate normal distribution, and we discuss maximum likelihood inference when a sample of independent clusters of varying sizes is available. We propose a method for maximum likelihood estimation which is an alternative to direct numerical maximization of the likelihood that sometimes exhibits non-convergence problems. We describe an algorithm for generating samples from the exchangeable multivariate Farlie–Gumbel–Morgenstern distribution with any marginals, using the structural properties of the distribution. Finally, we present the results of a simulation study designed to assess the properties of the maximum likelihood estimators, and we illustrate the use of the FGM distributions with the analysis of a small data set from a developmental toxicity study.     

Efficient designs for Bayesian networks with sub-tree bounds

We present upper and lower bounds for information measures, and use these to find the optimal design of experiments for Bayesian networks. The bounds are inspired by properties of the junction tree algorithm, which is commonly used for calculating conditional probabilities in graphical models like Bayesian networks. We demonstrate methods for iteratively improving the upper and lower bounds until they are sufficiently tight. We illustrate properties of the algorithm by tutorial examples in the case where we want to ensure optimality and for the case where the goal is an approximate solution with a guarantee. We further use the bounds to accelerate established algorithms for constructing useful designs. An example with petroleum fields in the North Sea is studied, where the design problem is related to exploration drilling campaigns. All of our examples consider binary random variables, but the theory can also be applied to other discrete or continuous distributions.     

Estimating the point of intersection of two lines under correlation

This paper introduces a new biostatistical approach to the problems of estimating true values and approximating the distribution of true values from unreliable data.We present the basic rationale for the unmixing method and report on a simulation study of its properties in estimating the centiles of a skewed, outlier-prone class of distributions.We also present an application to highly skewed USDA vitamin A intake data, and a pseudo-code version of the unmixing algorithm that we hope will allow other researchers to experiment with it     

An iterative algorithm for optimal variable weighting in K-means clustering

The K-means clustering method is a widely adopted clustering algorithm in data mining and pattern recognition, where the partitions are made by minimizing the total within group sum of squares based on a given set of variables. Weighted K-means clustering is an extension of the K-means method by assigning nonnegative weights to the set of variables. In this paper, we aim to obtain more meaningful and interpretable clusters by deriving the optimal variable weights for weighted K-means clustering. Specifically, we improve the weighted k-means clustering method by introducing a new algorithm to obtain the globally optimal variable weights based on the Karush-Kuhn-Tucker conditions. We present the mathematical formulation for the clustering problem, derive the structural properties of the optimal weights, and implement an recursive algorithm to calculate the optimal weights. Numerical examples on simulated and real data indicate that our method is superior in both clustering accuracy and computational efficiency.     

Local Likelihood Estimation in Generalized Additive Models

ABSTRACT.  Generalized additive models are a popular class of multivariate non-parametric regression models, due in large part to the ease of use of the local scoring estimation algorithm. However, the theoretical properties of the local scoring estimator are poorly understood. In this article, we propose a local likelihood estimator for generalized additive models that is closely related to the local scoring estimator fitted by local polynomial regression. We derive the statistical properties of the estimator and show that it achieves the same asymptotic convergence rate as a one-dimensional local polynomial regression estimator. We also propose a wild bootstrap estimator for calculating point-wise confidence intervals for the additive component functions. The practical behaviour of the proposed estimator is illustrated through a simulation experiment.     

Isotonic estimation of survival under a misattribution of cause of death

Several authors have indicated that incorrectly classified cause of death for prostate cancer survivors may have played a role in the observed recent peak and decline of prostate cancer mortality. Motivated by the suggestion we studied a competing risks model where other cause of death may be misattributed as a death of interest. We first consider a na?ve approach using unconstrained nonparametric maximum likelihood estimation (NPMLE), and then present the constrained NPMLE where the survival function is forced to be monotonic. Surprising observations were made as we studied their small-sample and asymptotic properties in continuous and discrete situations. Contrary to the common belief that the non-monotonicity of a survival function NPMLE is a small-sample problem, the constrained NPMLE is asymptotically biased in the continuous setting. Other isotonic approaches, the supremum (SUP) method and the Pooled-Adjacent-Violators (PAV) algorithm, and the EM algorithm are also considered. We found that the EM algorithm is equivalent to the constrained NPMLE. Both SUP method and PAV algorithm deliver consistent and asymptotically unbiased estimator. All methods behave well asymptotically in the discrete time setting. Data from the Surveillance, Epidemiology and End Results (SEER) database are used to illustrate the proposed estimators.     

The generalized Poisson-binomial distribution and the computation of its distribution function

The Poisson-binomial distribution is useful in many applied problems in engineering, actuarial science and data mining. The Poisson-binomial distribution models the distribution of the sum of independent but non-identically distributed random indicators whose success probabilities vary. In this paper, we extend the Poisson-binomial distribution to a generalized Poisson-binomial (GPB) distribution. The GPB distribution corresponds to the case where the random indicators are replaced by two-point random variables, which can take two arbitrary values instead of 0 and 1 as in the case of random indicators. The GPB distribution has found applications in many areas such as voting theory, actuarial science, warranty prediction and probability theory. As the GPB distribution has not been studied in detail so far, we introduce this distribution first and then derive its theoretical properties. We develop an efficient algorithm for the computation of its distribution function, using the fast Fourier transform. We test the accuracy of the developed algorithm by comparing it with enumeration-based exact method and the results from the binomial distribution. We also study the computational time of the algorithm under various parameter settings. Finally, we discuss the factors affecting the computational efficiency of the proposed algorithm and illustrate the use of the software package.     

On the Computation and Finite Sample Behavior of the Constrained M-Estimates for Multivariate Location and Scatter

Abstract

Constrained M (CM) estimates of multivariate location and scatter [Kent, J. T., Tyler, D. E. (1996). Constrained M-estimation for multivariate location and scatter. Ann. Statist. 24:1346–1370] are defined as the global minimum of an objective function subject to a constraint. These estimates combine the good global robustness properties of the S estimates and the good local robustness properties of the redescending M estimates. The CM estimates are not explicitly defined. Numerical methods have to be used to compute the CM estimates. In this paper, we give an algorithm to compute the CM estimates. Using the algorithm, we give a small simulation study to demonstrate the capability of the algorithm finding the CM estimates, and also to explore the finite sample behavior of the CM estimates. We also use the CM estimators to estimate the location and scatter parameters of some multivariate data sets to see the performance of the CM estimates dealing with the real data sets that may contain outliers.