Polynomial Histograms for Multivariate Density and Mode Estimation

Abstract. We consider the problem of efficiently estimating multivariate densities and their modes for moderate dimensions and an abundance of data. We propose polynomial histograms to solve this estimation problem. We present first‐ and second‐order polynomial histogram estimators for a general d‐dimensional setting. Our theoretical results include pointwise bias and variance of these estimators, their asymptotic mean integrated square error (AMISE), and optimal binwidth. The asymptotic performance of the first‐order estimator matches that of the kernel density estimator, while the second order has the faster rate of O(n^?6/(d+6)). For a bivariate normal setting, we present explicit expressions for the AMISE constants which show the much larger binwidths of the second order estimator and hence also more efficient computations of multivariate densities. We apply polynomial histogram estimators to real data from biotechnology and find the number and location of modes in such data.     

Approximate MLEs for the location and scale parameters of the skew logistic distribution

Azzalini (Scand J Stat 12:171–178, 1985) provided a methodology to introduce skewness in a normal distribution. Using the same method of Azzalini (1985), the skew logistic distribution can be easily obtained by introducing skewness to the logistic distribution. For the skew logistic distribution, the likelihood equations do not provide explicit solutions for the location and scale parameters. We present a simple method of deriving explicit estimators by approximating the likelihood equations appropriately. We examine numerically the bias and variance of these estimators and show that these estimators are as efficient as the maximum likelihood estimators (MLEs). The coverage probabilities of the pivotal quantities (for location and scale parameters) based on asymptotic normality are shown to be unsatisfactory, especially when the effective sample size is small. To improve the coverage probabilities and for constructing confidence intervals, we suggest the use of simulated percentage points. Finally, we present a numerical example to illustrate the methods of inference developed here.     

Estimating with percentile grouped and truncated date from a scale parameter distribution

Data which is grouped and truncated is considered. We are given numbers n₁<…<n_k=n and we observe X_ni ),i=1,…k, and the tottal number of observations available (N> n_k is unknown. If the underlying distribution has one unknown parameter θ which enters as a scale parameter, we examine the form of the equations for both conditional, unconditional and modified maximum likelihood estimators of θ and N and examine when these estimators will be finite, and unique. We also develop expressions for asymptotic bias and search for modified estimators which minimize the maximum asymptotic bias. These results are specialized tG the zxponential distribution. Methods of computing the solutions to the likelihood equatims are also discussed.     

Asymptotics of bivariate penalised splines

We study the class of bivariate penalised splines that use tensor product splines and a smoothness penalty. Similar to Claeskens, G., Krivobokova, T., and Opsomer, J.D. [(2009), ‘Asymptotic Properties of Penalised Spline Estimators’, Biometrika, 96(3), 529–544] for the univariate penalised splines, we show that, depending on the number of knots and penalty, the global asymptotic convergence rate of bivariate penalised splines is either similar to that of tensor product regression splines or to that of thin plate splines. In each scenario, the bivariate penalised splines are found rate optimal in the sense of Stone, C.J. [(12, 1982), ‘Optimal Global Rates of Convergence for Nonparametric Regression’, The Annals of Statistics, 10(4), 1040–1053] for a corresponding class of functions with appropriate smoothness. For the scenario where a small number of knots is used, we obtain expressions for the local asymptotic bias and variance and derive the point-wise and uniform asymptotic normality. The theoretical results are applicable to tensor product regression splines.     

Large sample behavior of the Bernstein copula estimator

Bernstein polynomial estimators have been used as smooth estimators for density functions and distribution functions. The idea of using them for copula estimation has been given in Sancetta and Satchell (2004). In the present paper we study the asymptotic properties of this estimator: almost sure consistency rates and asymptotic normality. We also obtain explicit expressions for the asymptotic bias and asymptotic variance and show the improvement of the asymptotic mean squared error compared to that of the classical empirical copula estimator. A small simulation study illustrates this superior behavior in small samples.     

Point and Interval Estimation for a Generalized Logistic Distribution Under Progressive Type II Censoring

The likelihood equations based on a progressively Type II censored sample from a Type I generalized logistic distribution do not provide explicit solutions for the location and scale parameters. We present a simple method of deriving explicit estimators by approximating the likelihood equations appropriately. We examine numerically the bias and variance of these estimators and show that these estimators are as efficient as the maximum likelihood estimators (MLEs). The probability coverages of the pivotal quantities (for location and scale parameters) based on asymptotic normality are shown to be unsatisfactory, especially when the effective sample size is small. Therefore we suggest using unconditional simulated percentage points of these pivotal quantities for the construction of confidence intervals. A wide range of sample sizes and progressive censoring schemes have been considered in this study. Finally, we present a numerical example to illustrate the methods of inference developed here.     

Estimation for semi-Markov models with partial observations via self-consistency equations

Nonparametric estimators of the survival function S(t) = P(T ≥ t) for a partially observed time variable T have been defined by several methods, in particular, by integral self-consistency equations. The author establishes explicit expressions of the estimators in an additive form and extend this approach to several cases: a left-truncated and right-censored variable and the left-censored or left-truncated sojourn times of a right-censored semi-Markov process. These estimators are always identical to the product-limit estimators if hazard functions may be defined.     

Bias-corrected maximum likelihood estimation of the parameters of the generalized Pareto distribution

ABSTRACT

We derive analytic expressions for the biases, to O(n^?1), of the maximum likelihood estimators of the parameters of the generalized Pareto distribution. Using these expressions to bias-correct the estimators in a selective manner is found to be extremely effective in terms of bias reduction, and can also result in a small reduction in relative mean squared error (MSE). In terms of remaining relative bias, the analytic bias-corrected estimators are somewhat less effective than their counterparts obtained by using a parametric bootstrap bias correction. However, the analytic correction out-performs the bootstrap correction in terms of remaining %MSE. It also performs credibly relative to other recently proposed estimators for this distribution. Taking into account the relative computational costs, this leads us to recommend the selective use of the analytic bias adjustment for most practical situations.     

A New Family of Nonparametric Quantile Estimators

A new core methodology for creating nonparametric L-quantile estimators is introduced and three new quantile L-estimators (SV1 _p , SV2 _p , and SV3 _p ) are constructed using the new methodology. Monte Carlo simulation was used in order to investigate the performance of the new estimators for small and large samples under normal distribution and a variety of light and heavy-tailed symmetric and asymmetric distributions. The new estimators outperform, in most of the cases studied, the Harrell–Davis quantile estimator and the weighted average at X _([np]) quantile estimator.     

Estimation in Epidemics with Incomplete Observations

The construction of estimating equations by martingale methods is generalized to yield estimators with explicit expressions for the parameters of the birth-and-death and the general epidemic processes when only partial observations are available. (For the birth-and-death process the death process is observed but the number of births is observed only at the end and for the general epidemic process only the removal process is observed.) For large populations, the use of the martingale central limit theorem yields asymptotic confidence regions for the parameters. Explicit expressions are derived for estimators of the variances of the large sample distributions. The range of validity and usefulness of the new estimators is determined by simulation.     

Estimating and testing group effects from type i censored normal samples in experimental design

The maximum likelihood (ML) equations calculated from censored normal samples do not admit explicit solutions. A principle of modification is given and modified maximum likelihood (MML) equations, which admit explicit solutions, are defined. This approach makes it possible to tackle the hitherto unresolved problem of estimating and testing hypotheses about group-effects in one-way classification experimental designs based on Type I censored normal samples. The MML estimators of group-effects are obtained as explicit functions of sample observations and shown to be asymptotically identical with the ML estimators and hence BAN (best asymptotic normal) estimators. A statistic t is defined to test a linear contrast of group-effects and shown to be asymptotically normally distributed. A numerical example is presented which illustrates the procedure.     

Estimating nonlinear additive models with nonstationarities and correlated errors

In this paper, we study a nonparametric additive regression model suitable for a wide range of time series applications. Our model includes a periodic component, a deterministic time trend, various component functions of stochastic explanatory variables, and an AR(p) error process that accounts for serial correlation in the regression error. We propose an estimation procedure for the nonparametric component functions and the parameters of the error process based on smooth backfitting and quasimaximum likelihood methods. Our theory establishes convergence rates and the asymptotic normality of our estimators. Moreover, we are able to derive an oracle‐type result for the estimators of the AR parameters: Under fairly mild conditions, the limiting distribution of our parameter estimators is the same as when the nonparametric component functions are known. Finally, we illustrate our estimation procedure by applying it to a sample of climate and ozone data collected on the Antarctic Peninsula.     

Estimation of the mixtures of GLMs with covariate-dependent mixing proportions

ABSTRACT

In this article, we study the estimation for a class of semiparametric mixtures of generalized linear models where mixing proportions depend on a covariate non parametrically. We investigate a backfitting estimation procedure and show the asymptotic normality of the proposed estimators under mild conditions. We conduct simulation to show the good performance of our methodology and give a real data analysis as an illustration.     

Robust Linear Calibration

We regard the simple linear calibration problem where only the response y of the regression line y = β₀ + β₁ t is observed with errors. The experimental conditions t are observed without error. For the errors of the observations y we assume that there may be some gross errors providing outlying observations. This situation can be modeled by a conditionally contaminated regression model. In this model the classical calibration estimator based on the least squares estimator has an unbounded asymptotic bias. Therefore we introduce calibration estimators based on robust one-step-M-estimators which have a bounded asymptotic bias. For this class of estimators we discuss two problems: The optimal estimators and their corresponding optimal designs. We derive the locally optimal solutions and show that the maximin efficient designs for non-robust estimation and robust estimation coincide.     

Bias Reduction for the Maximum Likelihood Estimator of the Parameters of the Generalized Rayleigh Family of Distributions

We derive analytic expressions for the biases, to O(n^{? 1}), of the maximum likelihood estimators of the parameters of the generalized Rayleigh distribution family. Using these expressions to bias-correct the estimators is found to be extremely effective in terms of bias reduction, and generally results in a small reduction in relative mean squared error. In general, the analytic bias-corrected estimators are also found to be superior to the alternative of bias-correction via the bootstrap.     

Asymptotic properties of one-step M-estimators

We study the asymptotic behavior of one-step M-estimators based on not necessarily independent identically distributed observations. In particular, we find conditions for asymptotic normality of these estimators. Asymptotic normality of one-step M-estimators is proven under a wide spectrum of constraints on the exactness of initial estimators. We discuss the question of minimal restrictions on the exactness of initial estimators. We also discuss the asymptotic behavior of the solution to an M-equation closest to the parameter under consideration. As an application, we consider some examples of one-step approximation of quasi-likelihood estimators in nonlinear regression.     

Asymptotic Theory for the Multiscale Wavelet Density Derivative Estimator

Abstract

In this paper we consider the wavelet-based estimation of density derivatives. The multiscale density derivative estimator is proposed which is constructed by using a number of scaling functions. Asymptotic theory is developed in which asymptotic expressions for the bias, the variance and the mean integrated squared error are included. In addition, asymptotic normality of the proposed estimator is proved. Theoretical and numerical comparisons with the usual kernel-based estimators are also reported.     

Double filter instrumental variable estimation of panel data models with weakly exogenous variables

In this article, we propose instrumental variables (IV) and generalized method of moments (GMM) estimators for panel data models with weakly exogenous variables. The model is allowed to include heterogeneous time trends besides the standard fixed effects (FE). The proposed IV and GMM estimators are obtained by applying a forward filter to the model and a backward filter to the instruments in order to remove FE, thereby called the double filter IV and GMM estimators. We derive the asymptotic properties of the proposed estimators under fixed T and large N, and large T and large N asymptotics where N and T denote the dimensions of cross section and time series, respectively. It is shown that the proposed IV estimator has the same asymptotic distribution as the bias corrected FE estimator when both N and T are large. Monte Carlo simulation results reveal that the proposed estimator performs well in finite samples and outperforms the conventional IV/GMM estimators using instruments in levels in many cases.     

Rare and clustered population estimation using the adaptive cluster sampling with some robust measures

The use of robust measures helps to increase the precision of the estimators, especially for the estimation of extremely skewed distributions. In this article, a generalized ratio estimator is proposed by using some robust measures with single auxiliary variable under the adaptive cluster sampling (ACS) design. We have incorporated tri-mean (TM), mid-range (MR) and Hodges-Lehman (HL) of the auxiliary variable as robust measures together with some conventional measures. The expressions of bias and mean square error (MSE) of the proposed generalized ratio estimator are derived. Two types of numerical study have been conducted using artificial clustered population and real data application to examine the performance of the proposed estimator over the usual mean per unit estimator under simple random sampling (SRS). Related results of the simulation study show that the proposed estimators provide better estimation results on both real and artificial population over the competing estimators.     

Bias reduction in kernel density estimation

In this paper, we propose two kernel density estimators based on a bias reduction technique. We study the properties of these estimators and compare them with Parzen–Rosenblatt's density estimator and Mokkadem, A., Pelletier, M., and Slaoui, Y. (2009, ‘The stochastic approximation method for the estimation of a multivariate probability density’, J. Statist. Plann. Inference, 139, 2459–2478) is density estimators. It turns out that, with an adequate choice of the parameters of the two proposed estimators, the rate of convergence of two estimators will be faster than the two classical estimators and the asymptotic MISE (Mean Integrated Squared Error) will be smaller than the two classical estimators. We corroborate these theoretical results through simulations.