Semi-parametric modelling and likelihood estimation with estimating equations

This paper proposes a semi-parametric modelling and estimating method for analysing censored survival data. The proposed method uses the empirical likelihood function to describe the information in data, and formulates estimating equations to incorporate knowledge of the underlying distribution and regression structure. The method is more flexible than the traditional methods such as the parametric maximum likelihood estimation (MLE), Cox's (1972) proportional hazards model, accelerated life test model, quasi-likelihood (Wedderburn, 1974) and generalized estimating equations (Liang & Zeger, 1986). This paper shows the existence and uniqueness of the proposed semi-parametric maximum likelihood estimates (SMLE) with estimating equations. The method is validated with known cases studied in the literature. Several finite sample simulation and large sample efficiency studies indicate that when the sample size is larger than 100 the SMLE is compatible with the parametric MLE; and in all case studies, the SMLE is about 15% better than the parametric MLE with a mis-specified underlying distribution.     

Nonparametric maximum-likelihood estimation of probability measures: existence and consistency

This paper formulates the nonparametric maximum-likelihood estimation of probability measures and generalizes the consistency result on the maximum-likelihood estimator (MLE). We drop the independent assumption on the underlying stochastic process and replace it with the assumption that the stochastic process is stationary and ergodic. The present proof employs Birkhoff's ergodic theorem and the martingale convergence theorem. The main result is applied to the parametric and nonparametric maximum-likelihood estimation of density functions.     

On maximum likelihood estimation of the long-memory parameter in fractional Gaussian noise

Approximate normality and unbiasedness of the maximum likelihood estimate (MLE) of the long-memory parameter H of a fractional Brownian motion hold reasonably well for sample sizes as small as 20 if the mean and scale parameter are known. We show in a Monte Carlo study that if the latter two parameters are unknown the bias and variance of the MLE of H both increase substantially. We also show that the bias can be reduced by using a parametric bootstrap procedure. In very large samples, maximum likelihood estimation becomes problematic because of the large dimension of the covariance matrix that must be inverted. To overcome this difficulty, we propose a maximum likelihood method based upon first differences of the data. These first differences form a short-memory process. We split the data into a number of contiguous blocks consisting of a relatively small number of observations. Computation of the likelihood function in a block then presents no computational problem. We form a pseudo-likelihood function consisting of the product of the likelihood functions in each of the blocks and provide a formula for the standard error of the resulting estimator of H. This formula is shown in a Monte Carlo study to provide a good approximation to the true standard error. The computation time required to obtain the estimate and its standard error from large data sets is an order of magnitude less than that required to obtain the widely used Whittle estimator. Application of the methodology is illustrated on two data sets.     

Quantile regression estimation of partially linear additive models

In this paper, we consider the estimation of partially linear additive quantile regression models where the conditional quantile function comprises a linear parametric component and a nonparametric additive component. We propose a two-step estimation approach: in the first step, we approximate the conditional quantile function using a series estimation method. In the second step, the nonparametric additive component is recovered using either a local polynomial estimator or a weighted Nadaraya–Watson estimator. Both consistency and asymptotic normality of the proposed estimators are established. Particularly, we show that the first-stage estimator for the finite-dimensional parameters attains the semiparametric efficiency bound under homoskedasticity, and that the second-stage estimators for the nonparametric additive component have an oracle efficiency property. Monte Carlo experiments are conducted to assess the finite sample performance of the proposed estimators. An application to a real data set is also illustrated.     

Variance estimation for tree-order restricted models

In this article, we discuss the estimation of the common variance of several normal populations with tree-order restricted means. We discuss the asymptotic properties of the maximum-likelihood estimator (MLE) of the variance as the number of populations tends to infinity. We consider several cases of various orders of the sample sizes and show that the MLE of the variance may or may not be consistent or be asymptotically normal.     

An alternative circular smoothing method to nonparametric estimation of periodic functions

This article provides alternative circular smoothing methods in nonparametric estimation of periodic functions. By treating the data as ‘circular’, we solve the “boundary issue” in the nonparametric estimation treating the data as ‘linear’. By redefining the distance metric and signed distance, we modify many estimators used in the situations involving periodic patterns. In the perspective of ‘nonparametric estimation of periodic functions’, we present the examples in nonparametric estimation of (1) a periodic function, (2) multiple periodic functions, (3) an evolving function, (4) a periodically varying-coefficient model and (5) a generalized linear model with periodically varying coefficient. In the perspective of ‘circular statistics’, we provide alternative approaches to calculate the weighted average and evaluate the ‘linear/circular–linear/circular’ association and regression. Simulation studies and an empirical study of electricity price index have been conducted to illustrate and compare our methods with other methods in the literature.     

Comparison of methods of estimation for parameters of generalized Poisson distribution through simulation study

In this article, we take a brief overview of different functional forms of generalized Poisson distribution (GPD) and various methods of its parameter estimation found in the literature. We compare the method of moment estimation (ME) and maximum likelihood estimation (MLE) of parameters of GPD through simulation study in terms of bias, MSE and covariance. To simulate random numbers from GPD, we develop a Matlab function gpoissrnd(). The simulation study leads to the important conclusion that the ME performs better or equally good as compared to MLE when sample size is small.

Further we fit the GPD to various datasets in literature using both estimation methods and observe that the results do not differ significantly even though the sample size is large. Overall, we conclude that for GPD, use of ME in place of MLE will lead to almost similar results. The computational simplicity in calculation of ME as compared to MLE also gives support to the use of ME in case of GPD for practitioners.     

Nonparametric maximum likelihood estimation for shifted curves

The analysis of a sample of curves can be done by self-modelling regression methods. Within this framework we follow the ideas of nonparametric maximum likelihood estimation known from event history analysis and the counting process set-up. We derive an infinite dimensional score equation and from there we suggest an algorithm to estimate the shape function for a simple shape invariant model. The nonparametric maximum likelihood estimator that we find turns out to be a Nadaraya–Watson-like estimator, but unlike in the usual kernel smoothing situation we do not need to select a bandwidth or even a kernel function, since the score equation automatically selects the shape and the smoothing parameter for the estimation. We apply the method to a sample of electrophoretic spectra to illustrate how it works.     

时变系数空间自回归面板数据模型的极大似然估计

本文对扰动项存在跨时期的异方差、但不存在序列相关的时变系数空间自回归模型提出了极大似然的估计方法,并证明了该估计量的一致性,同时,证明了该估计量渐进服从正态分布,由此说明该估计量具有优良的大样本性质。同时,我们还对本文所提出估计量的小样本性质进行了数值模拟。本文研究表明,估计量虽然在N较小时偏差较大,但是随着N的不断增加,估计量偏差减小,体现了比较优良的渐进性质。同时,估计量的偏差会随着时期数的增加而变大,这说明本文所提出的估计方法适用于个体数较多、时期数较少的短面板数据。     

Numerical maximum likelihood estimation for the g-and-k and generalized g-and-h distributions

Continuing increases in computing power and availability mean that many maximum likelihood estimation (MLE) problems previously thought intractable or too computationally difficult can now be tackled numerically. However, ML parameter estimation for distributions whose only analytical expression is as quantile functions has received little attention. Numerical MLE procedures for parameters of new families of distributions, the g-and-k and the generalized g-and-h distributions, are presented and investigated here. Simulation studies are included, and the appropriateness of using asymptotic methods examined. Because of the generality of these distributions, the investigations are not only into numerical MLE for these distributions, but are also an initial investigation into the performance and problems for numerical MLE applied to quantile-defined distributions in general. Datasets are also fitted using the procedures here. Results indicate that sample sizes significantly larger than 100 should be used to obtain reliable estimates through maximum likelihood.     

Comparison of estimation methods for the Weibull distribution

Weibull distributions have received wide ranging applications in many areas including reliability, hydrology and communication systems. Many estimation methods have been proposed for Weibull distributions. But there has not been a comprehensive comparison of these estimation methods. Most studies have focused on comparing the maximum likelihood estimation (MLE) with one of the other approaches. In this paper, we first propose an L-moment estimator for the Weibull distribution. Then, a comprehensive comparison is made of the following methods: the method of maximum likelihood estimation (MLE), the method of logarithmic moments, the percentile method, the method of moments and the method of L-moments.     

A Bayesian model for local smoothing in kernel density estimation

A new procedure is proposed for deriving variable bandwidths in univariate kernel density estimation, based upon likelihood cross-validation and an analysis of a Bayesian graphical model. The procedure admits bandwidth selection which is flexible in terms of the amount of smoothing required. In addition, the basic model can be extended to incorporate local smoothing of the density estimate. The method is shown to perform well in both theoretical and practical situations, and we compare our method with those of Abramson (The Annals of Statistics 10: 1217–1223) and Sain and Scott (Journal of the American Statistical Association 91: 1525–1534). In particular, we note that in certain cases, the Sain and Scott method performs poorly even with relatively large sample sizes.We compare various bandwidth selection methods using standard mean integrated square error criteria to assess the quality of the density estimates. We study situations where the underlying density is assumed both known and unknown, and note that in practice, our method performs well when sample sizes are small. In addition, we also apply the methods to real data, and again we believe our methods perform at least as well as existing methods.     

Modified cross-validation in density estimation

We propose a new modified (biased) cross-validation method for adaptively determining the bandwidth in a nonparametric density estimation setup. It is shown that the method provides consistent minimizers. Some simulation results are reported on which compare the small sample behavior of the new and the classical cross-validation selectors.     

Maximum likelihood estimation for incomplete multinomial data via the weaver algorithm

In a multinomial model, the sample space is partitioned into a disjoint union of cells. The partition is usually immutable during sampling of the cell counts. In this paper, we extend the multinomial model to the incomplete multinomial model by relaxing the constant partition assumption to allow the cells to be variable and the counts collected from non-disjoint cells to be modeled in an integrated manner for inference on the common underlying probability. The incomplete multinomial likelihood is parameterized by the complete-cell probabilities from the most refined partition. Its sufficient statistics include the variable-cell formation observed as an indicator matrix and all cell counts. With externally imposed structures on the cell formation process, it reduces to special models including the Bradley–Terry model, the Plackett–Luce model, etc. Since the conventional method, which solves for the zeros of the score functions, is unfruitful, we develop a new approach to establishing a simpler set of estimating equations to obtain the maximum likelihood estimate (MLE), which seeks the simultaneous maximization of all multiplicative components of the likelihood by fitting each component into an inequality. As a consequence, our estimation amounts to solving a system of the equality attainment conditions to the inequalities. The resultant MLE equations are simple and immediately invite a fixed-point iteration algorithm for solution, which is referred to as the weaver algorithm. The weaver algorithm is short and amenable to parallel implementation. We also derive the asymptotic covariance of the MLE, verify main results with simulations, and compare the weaver algorithm with an MM/EM algorithm based on fitting a Plackett–Luce model to a benchmark data set.     

Semi-Parametric Estimation for Forward–Backward Stochastic Differential Equations

In this article, we suggest a semi-parametric estimation for Forward–Backward Stochastic Differential Equations (FBSDE), with a linear generator. Both the nonparametric and parametric estimators are computationally feasible and the asymptotic properties are standard in the sense of normality. Although there is a plug-in nonparametric estimator in parametric estimation, the high order kernel, under-smoothing and bias correction are not required. Some simulation studies are also given to illustrate our methods.     

A bivariate density estimation method based on level crossings

This article introduces a method of nonparametric bivariate density estimation based on a bivariate sample level crossing function, which leads to the construction of a bivariate level crossing empirical distribution function (BLCEDF). An efficiency function for this BLCEDF relative to the classical empirical distribution function (EDF), is derived. The BLCEDF gives more efficient estimates than the EDF in the tails of any underlying continuous distribution, for both small and large sample sizes. On the basis of BLCEDF we define a bivariate level crossing kernel density estimator (BLCKDE) and study its properties. We apply the BLCEDF and BLCKDE for various distributions and provide results of simulations that confirm the theoretical properties. A real-world example is given.     

A new orthogonality-based estimation for varying-coefficient partially linear models

Varying coefficient partially linear models are usually used for longitudinal data analysis, and an interest is mainly to improve efficiency of regression coefficients. By the orthogonality estimation technology and the quadratic inference function method, we propose a new orthogonality-based estimation method to estimate parameter and nonparametric components in varying coefficient partially linear models with longitudinal data. The proposed procedure can separately estimate the parametric and nonparametric components, and the resulting estimators do not affect each other. Under some mild conditions, we establish some asymptotic properties of the resulting estimators. Furthermore, the finite sample performance of the proposed procedure is assessed by some simulation experiments.     

Estimation of the joint survival function for successive duration times

In incident cohort studies, survival data often include subjects who have experienced an initiate event but have not experienced a subsequent event at the calendar time of recruitment. During the follow-up periods, subjects may undergo a series of successive events. Since the second/third duration process becomes observable only if the first/second event has occurred, the data are subject to left-truncation and dependent censoring. In this article, using the inverse-probability-weighted (IPW) approach, we propose nonparametric estimators for the estimation of the joint survival function of three successive duration times. The asymptotic properties of the proposed estimators are established. The simple bootstrap methods are used to estimate standard deviations and construct interval estimators. A simulation study is conducted to investigate the finite sample properties of the proposed estimators.     

Estimation of reliability in multicomponent stress–strength based on two parameter exponentiated Weibull Distribution

In this research article, we estimate the multicomponent stress–strength reliability of a system when strength and stress variates are drawn from an exponentiated Weibull distribution with different shape parameters α?and?β, and common shape and scale parameters γ and λ, respectively. We estimate the parameters by using maximum likelihood estimation (MLE) and hence the estimate of reliability obtained applying the MLE method of estimation when samples are drawn from stress and strength distributions. The small sample comparison of the reliability estimates is made through Monte Carlo simulation.     

Implementing particle swarm optimization algorithm to estimate the mixture of two Weibull parameters with censored data

We present the maximum likelihood estimation (MLE) via particle swarm optimization (PSO) algorithm to estimate the mixture of two Weibull parameters with complete and multiple censored data. A simulation study is conducted to assess the performance of the MLE via PSO algorithm, quasi-Newton method and expectation-maximization (EM) algorithm for different parameter settings and sample sizes in both uncensored and censored cases. The simulation results showed that the PSO algorithm outperforms the quasi-Newton method and the EM algorithm in most cases regarding bias and root mean square errors. Two numerical examples are used to demonstrate the performance of our proposed method.