A Note on the Estimation of Binomial Probabilities

I am concerned with the admissibility under quadratic loss of certain estimators of binomial probabilities. The minimum variance unbiased estimator is shown to be admissible for Pr(X = 0) and Pr(X = n), but it is inadmissible for Pr(X = k), where 0 < k < n. An example is given of an admissible maximum likelihood estimator (MLE). It is conjectured that the MLE is always admissible.     

Sparse discriminant analysis based on estimation of posterior probabilities

ABSTRACT

Fisher's linear discriminant analysis (FLDA) is known as a method to find a discriminative feature space for multi-class classification. As a theory of extending FLDA to an ultimate nonlinear form, optimal nonlinear discriminant analysis (ONDA) has been proposed. ONDA indicates that the best theoretical nonlinear map for maximizing the Fisher's discriminant criterion is formulated by using the Bayesian a posterior probabilities. In addition, the theory proves that FLDA is equivalent to ONDA when the Bayesian a posterior probabilities are approximated by linear regression (LR). Due to some limitations of the linear model, there is room to modify FLDA by using stronger approximation/estimation methods. For the purpose of probability estimation, multi-nominal logistic regression (MLR) is more suitable than LR. Along this line, in this paper, we develop a nonlinear discriminant analysis (NDA) in which the posterior probabilities in ONDA are estimated by MLR. In addition, in this paper, we develop a way to introduce sparseness into discriminant analysis. By applying L1 or L2 regularization to LR or MLR, we can incorporate sparseness in FLDA and our NDA to increase generalization performance. The performance of these methods is evaluated by benchmark experiments using last_exam17 standard datasets and a face classification experiment.     

Numerical computation of rectangular bivariate and trivariate normal and t probabilities

Algorithms for the computation of bivariate and trivariate normal and t probabilities for rectangles are reviewed. The algorithms use numerical integration to approximate transformed probability distribution integrals. A generalization of Plackett's formula is derived for bivariate and trivariate t probabilities. New methods are described for the numerical computation of bivariate and trivariate t probabilities. Test results are provided, along with recommendations for the most efficient algorithms for single and double precision computations.     

Bayesian Variable Selection Under Collinearity

In this article, we highlight some interesting facts about Bayesian variable selection methods for linear regression models in settings where the design matrix exhibits strong collinearity. We first demonstrate via real data analysis and simulation studies that summaries of the posterior distribution based on marginal and joint distributions may give conflicting results for assessing the importance of strongly correlated covariates. The natural question is which one should be used in practice. The simulation studies suggest that posterior inclusion probabilities and Bayes factors that evaluate the importance of correlated covariates jointly are more appropriate, and some priors may be more adversely affected in such a setting. To obtain a better understanding behind the phenomenon, we study some toy examples with Zellner’s g-prior. The results show that strong collinearity may lead to a multimodal posterior distribution over models, in which joint summaries are more appropriate than marginal summaries. Thus, we recommend a routine examination of the correlation matrix and calculation of the joint inclusion probabilities for correlated covariates, in addition to marginal inclusion probabilities, for assessing the importance of covariates in Bayesian variable selection.     

A Characterization of a Matric-Variate Exponential Distribution

Suppose S is a positive definite m x m random matrix and S> Ω denotes the event that S—Ω is positive definite, Ω being a constant positive definite matrix. Under very mild regularity conditions, we show that the constraint Pr(S>Ω₁+Ω₂ | S>Ω₂) = Pr(S>Ω₁) implies that S has a Wishart distribution on m+ 1 degrees of freedom.     

Order statistics from trivariate normal and -distributions in terms of generalized skew-normal and skew- distributions

We consider here a generalization of the skew-normal distribution, GSN(λ₁,λ₂,ρ), defined through a standard bivariate normal distribution with correlation ρ, which is a special case of the unified multivariate skew-normal distribution studied recently by Arellano-Valle and Azzalini [2006. On the unification of families of skew-normal distributions. Scand. J. Statist. 33, 561–574]. We then present some simple and useful properties of this distribution and also derive its moment generating function in an explicit form. Next, we show that distributions of order statistics from the trivariate normal distribution are mixtures of these generalized skew-normal distributions; thence, using the established properties of the generalized skew-normal distribution, we derive the moment generating functions of order statistics, and also present expressions for means and variances of these order statistics.Next, we introduce a generalized skew-t_ν distribution, which is a special case of the unified multivariate skew-elliptical distribution presented by Arellano-Valle and Azzalini [2006. On the unification of families of skew-normal distributions. Scand. J. Statist. 33, 561–574] and is in fact a three-parameter generalization of Azzalini and Capitanio's [2003. Distributions generated by perturbation of symmetry with emphasis on a multivariate skew t distribution. J. Roy. Statist. Soc. Ser. B 65, 367–389] univariate skew-t_ν form. We then use the relationship between the generalized skew-normal and skew-t_ν distributions to discuss some properties of generalized skew-t_ν as well as distributions of order statistics from bivariate and trivariate t_ν distributions. We show that these distributions of order statistics are indeed mixtures of generalized skew-t_ν distributions, and then use this property to derive explicit expressions for means and variances of these order statistics.     

Simulation-assisted saddlepoint approximation

A general saddlepoint/Monte Carlo method to approximate (conditional) multivariate probabilities is presented. This method requires a tractable joint moment generating function (m.g.f.), but does not require a tractable distribution or density. The method is easy to program and has a third-order accuracy with respect to increasing sample size in contrast to standard asymptotic approximations which are typically only accurate to the first order.

The method is most easily described in the context of a continuous regular exponential family. Here, inferences can be formulated as probabilities with respect to the joint density of the sufficient statistics or the conditional density of some sufficient statistics given the others. Analytical expressions for these densities are not generally available, and it is often not possible to simulate exactly from the conditional distributions to obtain a direct Monte Carlo approximation of the required integral. A solution to the first of these problems is to replace the intractable density by a highly accurate saddlepoint approximation. The second problem can be addressed via importance sampling, that is, an indirect Monte Carlo approximation involving simulation from a crude approximation to the true density. Asymptotic normality of the sufficient statistics suggests an obvious candidate for an importance distribution.

The more general problem considers the computation of a joint probability for a subvector of random T, given its complementary subvector, when its distribution is intractable, but its joint m.g.f. is computable. For such settings, the distribution may be tilted, maintaining T as the sufficient statistic. Within this tilted family, the computation of such multivariate probabilities proceeds as described for the exponential family setting.     

Exact average coverage probabilities and confidence coefficients of confidence intervals for discrete distributions

For a confidence interval (L(X),U(X)) of a parameter θ in one-parameter discrete distributions, the coverage probability is a variable function of θ. The confidence coefficient is the infimum of the coverage probabilities, inf  _θ P _θ (θ∈(L(X),U(X))). Since we do not know which point in the parameter space the infimum coverage probability occurs at, the exact confidence coefficients are unknown. Beside confidence coefficients, evaluation of a confidence intervals can be based on the average coverage probability. Usually, the exact average probability is also unknown and it was approximated by taking the mean of the coverage probabilities at some randomly chosen points in the parameter space. In this article, methodologies for computing the exact average coverage probabilities as well as the exact confidence coefficients of confidence intervals for one-parameter discrete distributions are proposed. With these methodologies, both exact values can be derived.     

A self-validating numerical method for computation of central and non-central F probabilities and percentiles

A self-validating numerical method based on interval analysis for the computation of central and non-central F probabilities and percentiles is reported. The major advantage of this approach is that there are guaranteed error bounds associated with the computed values (or intervals), i.e. the computed values satisfy the user-specified accuracy requirements. The methodology reported in this paper can be adapted to approximate the probabilities and percentiles for other commonly used distribution functions.     

The ruin probabilities of a discrete time risk model with one-sided linear claim sizes and dependent risks

This article investigates the ruin probabilities of a discrete time risk model with dependent claim sizes and dependent relation between insurance risks and financial risks. The risk-free and risky investments of an insurer lead to stochastic discount factors {θ_n}_{n ? 1}. The claim sizes are assumed to follow a one-sided linear process with independent and identically distributed (i.i.d.) innovations {?_n}_{n ? 1}. The i.i.d. random pairs {(?_n, θ_n)}_{n ? 1} follow a common bivariate Sarmanov-dependent distribution. When the common distribution of the innovations is heavy tailed, we establish some asymptotic estimates for the ruin probabilities of this discrete time risk model.     

GEOMETRIC DECAY OF THE STEADY-STATE PROBABILITIES IN A QUASI-BIRTH-AND-DEATH PROCESS WITH A COUNTABLE NUMBER OF PHASES

A sufficient condition is proved for geometric decay of the steady-state probabilities in a quasi-birth-and-death process having a countable number of phases in each level. If there is a positive number η and positive vectors x = (x _i) and y = (y _j ) satisfying some equations and inequalities, the steady-state probability π _mi decays geometrically with rate η in the sense π _mi ～ cη ^m x _i as m → ∞. As an example, the result is applied to a two-queue system with shorter queue discipline.     

Tail Probability Ratios of Normal and Student’s t Distributions

The ratio of normal tail probabilities and the ratio of Student’s t tail probabilities have gained an increased attention in statistics and related areas. However, they are not well studied in the literature. In this paper, we systematically study the functional behaviors of these two ratios. Meanwhile, we explore their difference as well as their relationship. It is surprising that the two ratios behave very different to each other. Finally, we conclude the paper by conducting some lower and upper bounds for the two ratios.     

Small -sample comparison of the exact and symptotic upper tail probabilitiesof chi-squared goodness -of-fit statistics: pearson's x 2,likelihood ratio,and power-divergence statistic(λ = 2/3)

The exact and asymptotic upper tail probabilities ( α= .l0, .05, .01, .001) of the three chi-squared goodness-of-fit statistics Pearson's X ², likelihood ratioG ², and power-divergence statisticD ² (λ ) , with λ = 2/3, are compared numerically for simple null hypotheses not involving parameter estimation. Three types of such hypotheses were investigated (equal cell probabilities, proportional cell probabilities, some fixed small expectations together with some increasing large expectations) for the number of cells being between 3 and 15, and for sample sizes from 10 to 40, increasing by steps of one. Rating the relative accuracy of the chi-squared approximation in terms of ±10% and ±20% intervals around α led to the following conclusions: 1. Using G ² is not recommended. 2 . At the more relevant significance levels α = .10 and α = .05X ² should be preferred over D ². Solely in case of unequal cell probabilitiesD ² is the better choice at α = .O1 and α = .001. 3 . Yarnold's (1970; Journal of the Amerin Statistical Association, 65, 864-886) rule for the minimum expectation when using X ² ("If the number of cells k is 3 or more, and if r denotes the number of expectations less than 5, then the minimum expectation may be as small as 5r/k.") generalizes to D ²; it gives a good lower limit for the expected cell frequencies, however, when the number of cells is greater than 3. For k = 3 , even sample sizes over 15 may be insufficient.     

Upper bounds for ruin probabilities under model uncertainty

Abstract

In this paper, we investigate some ruin problems for risk models that contain uncertainties on both claim frequency and claim size distribution. The problems naturally lead to the evaluation of ruin probabilities under the so-called G-expectation framework. We assume that the risk process is described as a class of G-compound Poisson process, a special case of the G-Lévy process. By using the exponential martingale approach, we obtain the upper bounds for the two-sided ruin probability as well as the ruin probability involving investment. Furthermore, we derive the optimal investment strategy under the criterion of minimizing this upper bound. Finally, we conclude that the upper bound in the case with investment is less than or equal to the case without investment.     

On Confidence Intervals for Process Capability Indices in a One-Way Random Model

In this article, we investigated the bootstrap calibrated generalized confidence limits for process capability indices C _pk for the one-way random effect model. Also, we derived Bissell's approximation formula for the lower confidence limit using Satterthwaite's method and calculated its coverage probabilities and expected values. Then we compared it with standard bootstrap (SB) method and generalized confidence interval method. The simulation results indicate that the confidence limit obtained offers satisfactory coverage probabilities. The proposed method is illustrated with the help of simulation studies and data sets.     

Simultaneous Closeness of k-Records

In this article, we study the simultaneous Pitman closeness of upper (and lower) k-records to a common parameter of the parent distribution and obtain general expressions for the corresponding probabilities. Since the usual record values are contained in the k-records, the corresponding results for the usual records can be deduced as special cases. The results are then applied to location-scale families with an emphasis on population quantiles. Exact expressions are derived for some common distributions such as exponential and uniform. In each case, the simultaneous closest k-record to a population quantile is determined.     

The Moment Bound is Tighter than Chernoff's Bound for Positive Tail Probabilities

Chernoff's bound on P[X ? t] is used almost universally when a tight bound on tail probabilities is required. In this article we show that for all positive t and for all distributions, the moment bound is tighter than Chernoff's bound. By way of example, we demonstrate that the improvement is often substantial.     

Coherency conditions for finite exchangeable inference

Summary We idenify the invertible coherent functional relation between an array of asserted conditional probabilities and the probability distribution for the sum of events that are regarded exchangeably, in the regular case thatP(N _N+1 |S _N =a) ∈ (0, 1) for everya=0, 1, ...,N. The result is used to construct a useful algebraic and geometrical representation of all coherent inferences in the regular case, including those that are nonlinear in the sum of the conditioning events. The special case in which conditional probabilities mimic observed frequencies within (0, 1) receives an exact solution, which allows an easy interpretation of its surprising consequences. Finally, we introduce a new direction in research on prior opinion assessment that this approach, inverse to the usual one, suggests.     

Robust Confidence Intervals for the Bernoulli Parameter

Despite the simplicity of the Bernoulli process, developing good confidence interval procedures for its parameter—the probability of success p—is deceptively difficult. The binary data yield a discrete number of successes from a discrete number of trials, n. This discreteness results in actual coverage probabilities that oscillate with the n for fixed values of p (and with p for fixed n). Moreover, this oscillation necessitates a large sample size to guarantee a good coverage probability when p is close to 0 or 1.

It is well known that the Wilson procedure is superior to many existing procedures because it is less sensitive to p than any other procedures, therefore it is less costly. The procedures proposed in this article work as well as the Wilson procedure when 0.1 ≤p ≤ 0.9, and are even less sensitive (i.e., more robust) than the Wilson procedure when p is close to 0 or 1. Specifically, when the nominal coverage probability is 0.95, the Wilson procedure requires a sample size 1, 021 to guarantee that the coverage probabilities stay above 0.92 for any 0.001 ≤ min {p, 1 ?p} <0.01. By contrast, our procedures guarantee the same coverage probabilities but only need a sample size 177 without increasing either the expected interval width or the standard deviation of the interval width.     

Model based labeling for mixture models

Label switching is one of the fundamental problems for Bayesian mixture model analysis. Due to the permutation invariance of the mixture posterior, we can consider that the posterior of a m-component mixture model is a mixture distribution with m! symmetric components and therefore the object of labeling is to recover one of the components. In order to do labeling, we propose to first fit a symmetric m!-component mixture model to the Markov chain Monte Carlo (MCMC) samples and then choose the label for each sample by maximizing the corresponding classification probabilities, which are the probabilities of all possible labels for each sample. Both parametric and semi-parametric ways are proposed to fit the symmetric mixture model for the posterior. Compared to the existing labeling methods, our proposed method aims to approximate the posterior directly and provides the labeling probabilities for all possible labels and thus has a model explanation and theoretical support. In addition, we introduce a situation in which the “ideally” labeled samples are available and thus can be used to compare different labeling methods. We demonstrate the success of our new method in dealing with the label switching problem using two examples.