Parameter selection in modified histogram estimates

A multivariate modified histogram density estimate depending on a reference density g and a partition P has been proved to have good consistency properties according to several information theoretic criteria. Given an i.i.d. sample, we show how to select automatically both g and P so that the expected L ₁ error of the corresponding selected estimate is within a given constant multiple of the best possible error plus an additive term which tends to zero under mild assumptions. Our method is inspired by the combinatorial tools developed by Devroye and Lugosi [Devroye, L. and Lugosi, G., 2001, Combinatorial Methods in Density Estimation (New York, NY: Springer–Verlag)] and it includes a wide range of reference density and partition models. Results of simulations are also presented.     

On tests of symmetry,marginal homogeneity and quasi-symmetry in two-way contingency tables based on minimum φ-divergence estimator with constraints

The restricted minimum φ-divergence estimator, [Pardo, J.A., Pardo, L. and Zografos, K., 2002, Minimum φ-divergence estimators with constraints in multinomial populations. Journal of Statistical Planning and Inference, 104, 221–237], is employed to obtain estimates of the cell frequencies of an I×I contingency table under hypotheses of symmetry, marginal homogeneity or quasi-symmetry. The associated φ-divergence statistics are distributed asymptotically as chi-squared distributions under the null hypothesis. The new estimators and test statistics contain, as particular cases, the classical estimators and test statistics previously presented in the literature for the cited problems. A simulation study is presented, for the symmetry problem, to choose the best function φ₂ for estimation and the best function φ₁ for testing.     

Quasi-universal bandwidth selection for kernel density estimators

Let f?_n, h denote the kernel density estimate based on a sample of size n drawn from an unknown density f. Using techniques from L₂ projection density estimators, the author shows how to construct a data-driven estimator f?_n, h which satisfies This paper is inspired by work of Stone (1984), Devroye and Lugosi (1996) and Birge and Massart (1997).     

The information for the direction of dependence in l1 regression

Fitting a linear regression for a response variable by minimising the sum of absolute deviations, L₁ regression, may be viewed as a maximum likelihood procedure applied to the Laplace distribution. An interesting bivariate case is where the conditional distribution of the response X₂ given X₁ and the marginal distribution of the explanatory variable X₁ are both Laplace. In this context we show there is information to distinguish the direction of dependence between X₁ and X₂ from observations. That is we may distinguish the model in which X₁ is dependent on X₂ from that in which X₂ is dependent on X₁ This is not true for L₂ regression based on the Normal distribution.     

Sequential Fixed-Width Confidence Interval for a Function of the Parameters

Let X₁…, X_m and Y₁…, Y_n be two independent sequences of i.i.d. random variables with distribution functions F_x(.|θ) and F_y(. | φ) respectively. Let g(θ, φ) be a real-valued function of the unknown parameters θ and φ. The purpose of this paper is to suggest a sequential procedure which gives a fixed-width confidence interval for g(θ, φ) so that the coverage probability is approximately α (preas-signed). Certain asymptotic optimality properties of the sequential procedure are established. A Monte Carlo study is presented.     

Variable selection via penalized minimum φ-divergence estimation in logistic regression

We propose penalized minimum φ-divergence estimator for parameter estimation and variable selection in logistic regression. Using an appropriate penalty function, we show that penalized φ-divergence estimator has oracle property. With probability tending to 1, penalized φ-divergence estimator identifies the true model and estimates nonzero coefficients as efficiently as if the sparsity of the true model was known in advance. The advantage of penalized φ-divergence estimator is that it produces estimates of nonzero parameters efficiently than penalized maximum likelihood estimator when sample size is small and is equivalent to it for large one. Numerical simulations confirm our findings.     

On the mean L1-error in the heteroscedastic deconvolution problem with compactly supported noises

We study the heteroscedastic deconvolution problem when random noises have compactly supported densities. In this context, the Fourier transforms of the densities can vanish on the real line. We propose a truncated type of estimator for target density and derive the convergence rate of the mean L¹-error uniformly over a class of target densities. A lower bound for the mean L¹-error is also established. Some simulations will be given to illustrate the performance of the proposed estimator.     

Robust variable selection for the varying coefficient model based on composite L 1–L 2 regression

The varying coefficient model (VCM) is an important generalization of the linear regression model and many existing estimation procedures for VCM were built on L ₂ loss, which is popular for its mathematical beauty but is not robust to non-normal errors and outliers. In this paper, we address the problem of both robustness and efficiency of estimation and variable selection procedure based on the convex combined loss of L ₁ and L ₂ instead of only quadratic loss for VCM. By using local linear modeling method, the asymptotic normality of estimation is driven and a useful selection method is proposed for the weight of composite L ₁ and L ₂. Then the variable selection procedure is given by combining local kernel smoothing with adaptive group LASSO. With appropriate selection of tuning parameters by Bayesian information criterion (BIC) the theoretical properties of the new procedure, including consistency in variable selection and the oracle property in estimation, are established. The finite sample performance of the new method is investigated through simulation studies and the analysis of body fat data. Numerical studies show that the new method is better than or at least as well as the least square-based method in terms of both robustness and efficiency for variable selection.     

Two-Sample Tests Based on the Integrated Empirical Process

Abstract

Let X ₁, …, X _m and Y ₁, …, Y _n be independent random variables, where X ₁, …, X _m are i.i.d. with continuous distribution function (df) F, and Y ₁, …, Y _n are i.i.d. with continuous df G. For testing the hypothesis H ₀: F = G, we introduce and study analogues of the celebrated Kolmogorov–Smirnov and one- and two-sided Cramér-von Mises statistics that are functionals of a suitably integrated two-sample empirical process. Furthermore, we characterize those distributions for which the new tests are locally Bahadur optimal within the setting of shift alternatives.     

BAYESIAN SMOOTHING WITH OCCASIONAL JUMPS

Consider a sequence of independent random variables X ₁, X ₂,…,X _n observed at n equally spaced time points where X _i has a probability distribution which is known apart from the values of a parameter θ _i ∈ R which may change from observation to observation. We consider the problem of estimating θ = (θ₁, θ₂,…,θ _n ) given the observed values of X ₁, X ₂,…,X _n . The paper proposes a prior distribution for the parameters θ for which sets of parameter values exhibiting no change, or no change apart from a few sudden large changes, or lots of small changes, all have positive prior probability. Markov chain sampling may be used to calculate Bayes estimates of the parameters. We report the results of a Monte Carlo study based on Poisson distributed data which compares the Bayes estimator with estimators obtained using cubic splines and with estimators derived from the Schwarz criterion. We conclude that the Bayes method is preferable in a minimax sense since it never produces the disastrously large errors of the other methods and pays only a modest price for this degree of safety. All three methods are used to smooth mortality rates for oesophageal cancer in Irish males aged 65–69 over the period 1955 through 1994.     

Exponentiality test based on Renyi distance between equilibrium distributions

ABSTRACT

On the basis of Csiszar's φ-divergence discrimination information, we propose a measure of discrepancy between equilibriums associated with two distributions. Proving that a distribution can be characterized by associated equilibrium distribution, a Renyi distance of the equilibrium distributions is constructed that made us to propose an EDF-based goodness-of-fit test for exponential distribution. For comparing the performance of the proposed test, some well-known EDF-based tests and some entropy-based tests are considered. Based on the simulation results, the proposed test has better powers than those of competing entropy-based tests for the alternatives with decreasing hazard rate function. The use of the proposed test is evaluated in an illustrative example.     

On the asymptotic normality of the L1- and L2-errors in histogram density estimation

The L₁ and L₂-errors of the histogram estimate of a density f from a sample X₁,X₂,…,Xn using a cubic partition are shown to be asymptotically normal without any unnecessary conditions imposed on the density f. The asymptotic variances are shown to depend on f only through the corresponding norm of f. From this follows the asymptotic null distribution of a goodness-of-fit test based on the total variation distance, introduced by Györfi and van der Meulen (1991). This note uses the idea of partial inversion for obtaining characteristic functions of conditional distributions, which goes back at least to Bartlett (1938).     

An extension of almost sure central limit theorem for self-normalized products of sums for mixing sequences

ABSTRACT

Let X, X₁, X₂, … be a sequence of strictly stationary φ-mixing random variables with EX = μ > 0. In this paper, we show that a self-normalized version of almost sure central limit theorem (ASCLT) holds under the assumptions that the mixing coefficients satisfy ∑^∞_{n = 1}φ^1/2(2ⁿ) < ∞ and the weight sequence {d_k} satisfies a mild growth condition similar to Kolmogorov’s condition for the LIL. This shows that logarithmic averages, used traditionally in ASCLT for products of sums, can be replaced by other averages, leading to considerably sharper results.     

THE CONVERGENCE RATE OF SEQUENTIAL FIXED-WIDTH CONFIDENCE INTERVALS FOR REGULAR FUNCTIONALS

Let X₁X₂,.be i.i.d. random variables and let U_n= (n r)^-1S?^(n,r) h (X_i1,., X_ir,) be a U-statistic with EU_n= v, v unknown. Assume that g(X₁) =E[h(X₁,.,X_r) - v |X₁]has a strictly positive variance s?². Further, let a be such that φ(a) - φ(-a) =α for fixed α, 0 < α < 1, where φ is the standard normal d.f., and let S²_n be the Jackknife estimator of n Var U_n. Consider the stopping times N(d)= min {n: S²_n: + n^-1≤²a^-2},d > 0, and a confidence interval for v of length 2d,of the form I_n,d= [U_n,-d, Un + d]. We assume that Var U_n is unknown, and hence, no fixed sample size method is available for finding a confidence interval for v of prescribed width 2d and prescribed coverage probability α Turning to a sequential procedure, let I_N(d),d be a sequence of sequential confidence intervals for v. The asymptotic consistency of this procedure, i.e. lim_{d → 0}P(v ∈ I_N(d),d)=α follows from Sproule (1969). In this paper, the rate at which |P(v ∈ I_N(d),d) converges to α is investigated. We obtain that |P(v ∈ I_N(d),d) - α| = 0 (d^{1/2-(1+k)/2(1+m)}), d → 0, where K = max {0,4 - m}, under the condition that E|h(X₁, X_r)|^m < ∞m > 2. This improves and extends recent results of Ghosh & DasGupta (1980) and Mukhopadhyay (1981).     

A comparison of confidence intervals generated by the scheffé and bonferroni methods

Simultaneous confidence intervals for the p means of a multivariate normal random variable with known variances may be generated by the projection method of Scheffé and by the use of Bonferroni's inequality. It has been conjectured that the Bonferroni intervals are shorter than the Scheffé intervals, at least for the usual confidence levels. This conjecture is proved for all p≥2 and all confidence levels above 50%. It is shown, incidentally, that for all p≥2 Scheffé's intervals are shorter for sufficiently small confidence levels. The results are also applicable to the Bonferroni and Scheffé intervals generated for multinomial proportions.     

On some pre-test and Stein-rule phi-divergence test estimators in the independence model of categorical data

For the model of independence in a two way contingency table, shrinkage estimators based on minimum φ$\phi$-divergence estimators and φ$\phi$-divergence statistics are considered. These estimators are based on the James–Stein-type rule and incorporate the idea of preliminary test estimator. The asymptotic bias and risk are obtained under contiguous alternative hypotheses, and on the basis of them a comparison study is carried out.     

Asymptotic expansion of the risk of maximum likelihood estimator with respect to α-divergence

For a given parametric probability model, we consider the risk of the maximum likelihood estimator with respect to α-divergence, which includes the special cases of Kullback–Leibler divergence, the Hellinger distance, and essentially χ²-divergence. The asymptotic expansion of the risk is given with respect to sample sizes up to order n^{? 2}. Each term in the expansion is expressed with the geometrical properties of the Riemannian manifold formed by the parametric probability model.     

Bayesian inference and diagnostics in zero-inflated generalized power series regression model

ABSTRACT

The paper provides a Bayesian analysis for the zero-inflated regression models based on the generalized power series distribution. The approach is based on Markov chain Monte Carlo methods. The residual analysis is discussed and case-deletion influence diagnostics are developed for the joint posterior distribution, based on the ψ-divergence, which includes several divergence measures such as the Kullback–Leibler, J-distance, L₁ norm, and χ²-square in zero-inflated general power series models. The methodology is reflected in a data set collected by wildlife biologists in a state park in California.     

Moment Consistency of the Exchangeably Weighted Bootstrap for Semiparametric M‐estimation

The bootstrap variance estimate is widely used in semiparametric inferences. However, its theoretical validity is a well‐known open problem. In this paper, we provide a first theoretical study on the bootstrap moment estimates in semiparametric models. Specifically, we establish the bootstrap moment consistency of the Euclidean parameter, which immediately implies the consistency of t‐type bootstrap confidence set. It is worth pointing out that the only additional cost to achieve the bootstrap moment consistency in contrast with the distribution consistency is to simply strengthen the L₁ maximal inequality condition required in the latter to the L_p maximal inequality condition for p≥1. The general L_p multiplier inequality developed in this paper is also of independent interest. These general conclusions hold for the bootstrap methods with exchangeable bootstrap weights, for example, non‐parametric bootstrap and Bayesian bootstrap. Our general theory is illustrated in the celebrated Cox regression model.     

Multiparameter estimation in truncated power series distributions under the stein's loss

LetX ₁,…,X _p be p(≥2)independent random variables, where each X.has a distribution belonging to a one parameter truncated power series

distribution. The problem is to estimate simultaneously the unknown parameters under asymmetric loss developed by James and Stein (Proc. Fourth Berkeley Symp. Math. Statist. Prob. 1, 361-380). Several new classes of dominating estimators are obtained by solving a certain difference inequality.