A target distribution model for nonparametric density estimation

In this paper a model is proposed which represents a wide class of continuous distributions. It is shown how the parameters of this model can be estimated leading to a distribution estimator and a corresponding density estimator. An important property of this estimator is that it can be structured to reflect a priori knowledge of the unknown distribution.

Finally, some examples are shown and some comparisons made with kernel and orthogonal series estimators.     

A method of moments estimator of tail dependence in meta-elliptical models

A meta-elliptical model is a distribution function whose copula is that of an elliptical distribution. The tail dependence function in such a bivariate model has a parametric representation with two parameters: a tail parameter and a correlation parameter. The correlation parameter can be estimated by robust methods based on the whole sample. Using the estimated correlation parameter as plug-in estimator, we then estimate the tail parameter applying a modification of the method of moments approach proposed in the paper by Einmahl et al. (2008). We show that such an estimator is consistent and asymptotically normal. Further, we derive the joint limit distribution of the estimators of the two parameters. We illustrate the small sample behavior of the estimator of the tail parameter by a simulation study and on real data, and we compare its performance to that of the competitive estimators.     

Comparison of renewal function estimators for censored data

The computation of the renewal function when the distribution function is completely known has received much attention in the literature. However, in many cases the form of the distribution function is unknown and has to be estimated nonparametrically. A nonparametric estimator for the renewal function for complete data was suggested by Frees (1986). In many cases, however, censoring of the lifetime might occur. We shall present parametric and nonparametric estimators of the renewal function based on censored data. In a simulation study we compare the nonparametric estimators with parametric estimators for the Weibull and lognormal distribution. The study suggests that the nonparametric estimator is a viable alternative to the parametric estimators when the lifetime distribution is unknown. Also, the nonparametric estimator is computationally simpler than the parametric estimator.     

Quantile regression in functional linear semiparametric model

This paper proposes nonparametric estimation methods for functional linear semiparametric quantile regression, where the conditional quantile of the scalar responses is modelled by both scalar and functional covariates and an additional unknown nonparametric function term. The slope function is estimated using the functional principal component basis and the nonparametric function is approximated by a piecewise polynomial function. The asymptotic distribution of the estimators of slope parameters is derived and the global convergence rate of the quantile estimator of unknown slope function is established under suitable norm. The asymptotic distribution of the estimator of the unknown nonparametric function is also established. Simulation studies are conducted to investigate the finite-sample performance of the proposed estimators. The proposed methodology is demonstrated by analysing a real data from ADHD-200 sample.     

Improved distribution quantile estimation

A technique for estimating the quantiles or percentiles of a distribution is developed. The parametric form of the distribution is assumed unknown. The estimation procedure is based on a kernel estimator of a probability density function and on aquantile estimator suggested by Harrell and Davis (1982). Simulation studies show that estimation of quantiles in moderately heavyto heavy tails of a distribution is substantially improved by use of the technique.     

The density function and the MSE dominance of the pre-test estimator in a heteroscedastic linear regression model with omitted variables

In this paper, we consider a heteroscedastic linear regression model with omitted variables. We derive the density function of the pre-test estimator consisting of the two-stage Aitken estimator (2SAE) and the ordinary least squares estimator (OLSE) after the pre-test for homoscedasticity. We also derive the first two moments based on the density function and show the sufficient condition for the pre-test estimator to dominate the 2SAE in terms of the MSE. Our numerical evaluations show that when this sufficient condition does not hold and when the magnitude of the specification error is large, the pre-test estimator can be dominated by the 2SAE, and further, the 2SAE can be dominated by the OLSE.     

Local Smoothing for Kernel Distribution Function Estimation

The problem of bandwidth selection for kernel-based estimation of the distribution function (cdf) at a given point is considered. With appropriate bandwidth, a kernel-based estimator (kdf) is known to outperform the empirical distribution function. However, such a bandwidth is unknown in practice. In pointwise estimation, the appropriate bandwidth depends on the point where the function is estimated. The existing smoothing methods use one common bandwidth to estimate the cdf. The accuracy of the resulting estimates varies substantially depending on the cdf and the point where it is estimated. We propose to select bandwidth by minimizing a bootstrap estimator of the MSE of the kdf. The resulting estimator performs reliably, irrespective of where the cdf is estimated. It is shown to be consistent under i.i.d. as well as strongly mixing dependence assumption. Two applications of the proposed estimator are shown in finance and seismology. We report a dataset on the S & P Nifty index values.     

Logspline Deconvolution in Besov Space

ABSTRACT. In this paper we consider logspline density estimation for random variables which are contaminated with random noise. In the logspline density estimation for data without noise, the logarithm of an unknown density function is estimated by a polynomial spline, the unknown parameters of which are given by maximum likelihood. When noise is present, B-splines and the Fourier inversion formula are used to construct the logspline density estimator of the unknown density function. Rates of convergence are established when the log-density function is assumed to be in a Besov space. It is shown that convergence rates depend on the smoothness of the density function and the decay rate of the characteristic function of the noise. Simulated data are used to show the finite-sample performance of inference based on the logspline density estimation.     

The optimal Lp norm estimator in linear regression models

The least squares estimator is usually applied when estimating the parameters in linear regression models. As this estimator is sensitive to departures from normality in the residual distribution, several alternatives have been proposed. The L_p norm estimators is one class of such alternatives. It has been proposed that the kurtosis of the residual distribution be taken into account when a choice of estimator in the L_p norm class is made (i.e. the choice of p). In this paper, the asymtotic variance of the estimators is used as the criterion in the choice of p. It is shown that when this criterion is applied, other characteristics of the residual distribution than the kurtosis (namely moments of order p-2 and 2p-2) are important.     

Robust Estimation in Binary Choice Models

The binary-response smoothed maximum score (SMS) estimator accommodates heteroskedasticity of an unknown form, but it may be heavily biased when the conditional error density is not differentiable or not bell shaped. We construct a new combined SMS estimator as a linear combination of individual estimators with weights chosen to minimize the trace of estimated mean squared error. This estimator is robust and rate-adaptive under weak assumptions on the density. Results of a Monte Carlo study confirm good performance of the combined estimator.     

A semiparametric estimator of the distribution function of a variable measured with error

The estimation of the distribution functon of a random variable X measured with error is studied. Let the i-th observation on X be denoted by Y_iX_i+ε_i where ε_i is the measuremen error. Let {Y_i} (i=1,2,…,n) be a sample of independent observations. It is assumed that {X_i} and {∈_i} are mutually independent and each is identically distributed. As is standard in the literature for this problem, the distribution of e is assumed known in the development of the methodology. In practice, the measurement error distribution is estimated from replicate observations.

The proposed semiparametric estimator is derived by estimating the quantises of X on a set of n transformed V-values and smoothing the estimated quantiles using a spline function. The number of parameters of the spline function is determined by the data with a simple criterion, such as AIC. In a simulation study, the semiparametric estimator dominates an optimal kernel estimator and a normal mixture estimator for a wide class of densities.

The proposed estimator is applied to estimate the distribution function of the mean pH value in a field plot. The density function of the measurement error is estimated from repeated measurements of the pH values in a plot, and is treated as known for the estimation of the distribution function of the mean pH value.     

Estimation of the Moment Generating Function

A number of statistical problems use the moment generating function (mgf) for purposes other than determining the moments of a distribution. If the distribution is not completely specified, then the mgf must be estimated from available data. The empirical mgf makes no assumptions concerning the underlying distribution except for the existence of the mgf. In contrast to the nonparametric approach provided by the empirical mgf, alternative estimators can be formed based on an assumed parametric model. Comparison of these approaches is considered for two parametric models; the normal and a one parameter gamma. Comparison criteria are efficiency and empirical confidence interval coverage. In general the parametric estimators outperform the empirical mgf when the model is correct. The comparisons are extended to underlying models which are two component mixtures from the distributional family assumed by the parametric estimators. Under the mixture models the superiority of the parametric estimator depends upon the model, value of the argument of the mgf, and the comparison criterion. The empirical mgf is the better estimator in some cases.     

Lp-estimators in ARCH models

For ergodic ARCH processes, we introduce a one-parameter family of L_p-estimators. The construction is based on the concept of weighted M-estimators. Under weak assumptions on the error distribution, the consistency is established. The asymptotic normality is proved for the special cases p=1 and 2. To prove the asymptotic normality of the L₁-estimator, one needs the existence of a density of the squares of the errors, whereas for the L₂-estimator the existence of fourth moments is assumed. The asymptotic covariance matrix of the estimator depends on the unknown parameter which can be substituted by consistent estimators. For the L₁-estimator we construct a kernel estimator for the unknown density of the square of the errors.     

Asymptotics of cross-validated risk estimation in estimator selection and performance assessment

Risk estimation is an important statistical question for the purposes of selecting a good estimator (i.e., model selection) and assessing its performance (i.e., estimating generalization error). This article introduces a general framework for cross-validation and derives distributional properties of cross-validated risk estimators in the context of estimator selection and performance assessment. Arbitrary classes of estimators are considered, including density estimators and predictors for both continuous and polychotomous outcomes. Results are provided for general full data loss functions (e.g., absolute and squared error, indicator, negative log density). A broad definition of cross-validation is used in order to cover leave-one-out cross-validation, V-fold cross-validation, Monte Carlo cross-validation, and bootstrap procedures. For estimator selection, finite sample risk bounds are derived and applied to establish the asymptotic optimality of cross-validation, in the sense that a selector based on a cross-validated risk estimator performs asymptotically as well as an optimal oracle selector based on the risk under the true, unknown data generating distribution. The asymptotic results are derived under the assumption that the size of the validation sets converges to infinity and hence do not cover leave-one-out cross-validation. For performance assessment, cross-validated risk estimators are shown to be consistent and asymptotically linear for the risk under the true data generating distribution and confidence intervals are derived for this unknown risk. Unlike previously published results, the theorems derived in this and our related articles apply to general data generating distributions, loss functions (i.e., parameters), estimators, and cross-validation procedures.     

Adaptive nonparametric density estimation with missing observations

It is well known that if some observations in a sample from the probability density are not available, then in general the density cannot be estimated. A possible remedy is to use an auxiliary variable that explains the missing mechanism. For this setting a data-driven estimator is proposed that mimics performance of an oracle that knows all observations from the sample. It is also proved that the estimator adapts to unknown smoothness of the density and its mean integrated squared error converges with a minimax rate. A numerical study, together with the analysis of a real data, shows that the estimator is feasible for small samples.     

Confidence Intervals for Current Status Data

Abstract.  The likelihood ratio statistic for testing pointwise hypotheses about the survival time distribution in the current status model can be inverted to yield confidence intervals (CIs). One advantage of this procedure is that CIs can be formed without estimating the unknown parameters that figure in the asymptotic distribution of the maximum likelihood estimator (MLE) of the distribution function. We discuss the likelihood ratio-based CIs for the distribution function and the quantile function and compare these intervals to several different intervals based on the MLE. The quantiles of the limiting distribution of the MLE are estimated using various methods including parametric fitting, kernel smoothing and subsampling techniques. Comparisons are carried out both for simulated data and on a data set involving time to immunization against rubella. The comparisons indicate that the likelihood ratio-based intervals are preferable from several perspectives.     

Simple expressions for a bivariate chisquare distribution

The joint distribution of the estimated variances from a correlated bivariate normal distribution has a long history. However, its joint probability density function, conditional moments and product moments are only known as infinite series. In this paper, simpler expressions, mostly finite sums of elementary functions, are derived for these properties. Expressions are also derived for the joint moment generating function and the joint characteristic function.     

A MORE GENERAL CRITERION FOR SUBSET SELECTION IN MULTIPLE LINEAR REGRESSION

ABSTRACT

In this article, we propose a more general criterion called S_p -criterion, for subset selection in the multiple linear regression Model. Many subset selection methods are based on the Least Squares (LS) estimator of β, but whenever the data contain an influential observation or the distribution of the error variable deviates from normality, the LS estimator performs ‘poorly’ and hence a method based on this estimator (for example, Mallows’ C_p -criterion) tends to select a ‘wrong’ subset. The proposed method overcomes this drawback and its main feature is that it can be used with any type of estimator (either the LS estimator or any robust estimator) of β without any need for modification of the proposed criterion. Moreover, this technique is operationally simple to implement as compared to other existing criteria. The method is illustrated with examples.     

The general expressions for the moments of lawless and wang's ordinary ridge regression estimator

In this paper, we derive the exact general expressions for the moments of an ordinary ridge regression (ORR) estimator for individual regression coefficients in a different way from Firinguetti (1987). Using the derived expressions, we evaluate numerically the first four moments of the ORR estimator, and examine its bias, mean square error, skewness and kurtosis. Further, Monte Carlo experiments are carried out in order to examine the shape of the density function of the ORR estimator.     

Moment Information for Probability Distributions,Without Solving the Moment Problem. I: Where is the Mode?

How much information does a small number of moments carry about the unknown distribution function? Is it possible to explicitly obtain from these moments some useful information, e.g., about the support, the modality, the general shape, or the tails of a distribution, without going into a detailed numerical solution of the moment problem? In this paper a theoretical result of Johnson and Rogers is generalized to be valid for all moment problems and is exploited to demonstrate that a few moments are able to provide us with valuable information about the position of the mode of an unknown (unimodal) distribution.