Smooth Nonparametric Allocation of Classification

A nonparametric discriminant analysis procedure that is robust to deviations from the usual assumptions is proposed. The procedure uses the projection pursuit methodology where the projection index is the two-group transvariation probability. We use allocation based on the centrality of the new point measured using a smooth version of point-group transvariation. It is shown that the new procedure provides lower misclassification error rates than competing methods for data from skewed heavy-tailed and skewed distributions as well as unequal training data sizes.     

Mutiple three-decision procedure for comparing several exponential treatments with the best control

In this paper, the three-decision procedures to classify p treatments as better than or worse than one control, proposed for normal/symmetric probability models [Bohrer, Multiple three-decision rules for parametric signs. J. Amer. Statist. Assoc. 74 (1979), pp. 432–437; Bohrer et al., Multiple three-decision rules for factorial simple effects: Bonferroni wins again!, J. Amer. Statist. Assoc. 76 (1981), pp. 119–124; Liu, A multiple three-decision procedure for comparing several treatments with a control, Austral. J. Statist. 39 (1997), pp. 79–92 and Singh and Mishra, Classifying logistic populations using sample medians, J. Statist. Plann. Inference 137 (2007), pp. 1647–1657]; in the literature, have been extended to asymmetric two-parameter exponential probability models to classify p(p≥1) treatments as better than or worse than the best of q(q≥1) control treatments in terms of location parameters. Critical constants required for the implementation of the proposed procedure are tabulated for some pre-specified values of probability of no misclassification. Power function of the proposed procedure is also defined and a common sample size necessary to guarantee various pre-specified power levels are tabulated. Optimal allocation scheme is also discussed. Finally, the implementation of the proposed methodology is demonstrated through numerical example.     

Evaluation of Gaussian orthant probabilities based on orthogonal projections to subspaces

In this paper, a new procedure is described for evaluating the probability that all elements of a normally distributed vector are non-negative, which is called the non-centered orthant probability. This probability is defined by a multivariate integral of the density function. The definition is simple, and this probability arises frequently in statistics because the normal distribution is prevalent. The method for evaluating this probability, however, is not obvious, because applying direct numerical integration is not practical except in low dimensional cases. In the procedure proposed in this paper, the problem is decomposed into sub-problems of lower dimension. Considering the projection onto subspaces, the solutions of the sub-problems can be shared in the evaluation of higher dimensional problems. Thus the sub-problems form a lattice structure. This reduces the computational time from a factorial order, where the interim results are not shared, to order \(p^{2}2^{p}\), which is faster than the procedures that have been reported in the literature.     

An Optimal Semiparametric Method for Two‐group Classification

In the classical discriminant analysis, when two multivariate normal distributions with equal variance–covariance matrices are assumed for two groups, the classical linear discriminant function is optimal with respect to maximizing the standardized difference between the means of two groups. However, for a typical case‐control study, the distributional assumption for the case group often needs to be relaxed in practice. Komori et al. (Generalized t ‐statistic for two‐group classification. Biometrics 2015, 71: 404–416) proposed the generalized t ‐statistic to obtain a linear discriminant function, which allows for heterogeneity of case group. Their procedure has an optimality property in the class of consideration. We perform a further study of the problem and show that additional improvement is achievable. The approach we propose does not require a parametric distributional assumption on the case group. We further show that the new estimator is efficient, in that no further improvement is possible to construct the linear discriminant function more efficiently. We conduct simulation studies and real data examples to illustrate the finite sample performance and the gain that it produces in comparison with existing methods.     

Sparse discriminant analysis based on estimation of posterior probabilities

ABSTRACT

Fisher's linear discriminant analysis (FLDA) is known as a method to find a discriminative feature space for multi-class classification. As a theory of extending FLDA to an ultimate nonlinear form, optimal nonlinear discriminant analysis (ONDA) has been proposed. ONDA indicates that the best theoretical nonlinear map for maximizing the Fisher's discriminant criterion is formulated by using the Bayesian a posterior probabilities. In addition, the theory proves that FLDA is equivalent to ONDA when the Bayesian a posterior probabilities are approximated by linear regression (LR). Due to some limitations of the linear model, there is room to modify FLDA by using stronger approximation/estimation methods. For the purpose of probability estimation, multi-nominal logistic regression (MLR) is more suitable than LR. Along this line, in this paper, we develop a nonlinear discriminant analysis (NDA) in which the posterior probabilities in ONDA are estimated by MLR. In addition, in this paper, we develop a way to introduce sparseness into discriminant analysis. By applying L1 or L2 regularization to LR or MLR, we can incorporate sparseness in FLDA and our NDA to increase generalization performance. The performance of these methods is evaluated by benchmark experiments using last_exam17 standard datasets and a face classification experiment.     

A proposal for plotting positions in probability plots

Probability plots allow us to determine whether a set of sample observations is distributed according to a theoretical distribution. Plotting positions are fundamental elements in statistics and, in particular, for the construction of probability plots. In this paper, a new plotting position to construct different probability plots, such as Q–Q Plot, P–P Plot and S–P Plot, is proposed. The proposed definition is based on the median of the ith order statistic of the theoretical distribution considered. The main feature of this plotting position formula is that it is independent of the theoretical distribution selected. Moreover, the procedure developed is ‘almost’ exact, reaching, without a high cost of time, an accuracy as great as we want, which avoids using approximations (proposed by other authors).     

Robust Model-Free Multiclass Probability Estimation

Classical statistical approaches for multiclass probability estimation are typically based on regression techniques such as multiple logistic regression, or density estimation approaches such as linear discriminant analysis (LDA) and quadratic discriminant analysis (QDA). These methods often make certain assumptions on the form of probability functions or on the underlying distributions of subclasses. In this article, we develop a model-free procedure to estimate multiclass probabilities based on large-margin classifiers. In particular, the new estimation scheme is employed by solving a series of weighted large-margin classifiers and then systematically extracting the probability information from these multiple classification rules. A main advantage of the proposed probability estimation technique is that it does not impose any strong parametric assumption on the underlying distribution and can be applied for a wide range of large-margin classification methods. A general computational algorithm is developed for class probability estimation. Furthermore, we establish asymptotic consistency of the probability estimates. Both simulated and real data examples are presented to illustrate competitive performance of the new approach and compare it with several other existing methods.     

Robust Confidence Intervals for the Bernoulli Parameter

Despite the simplicity of the Bernoulli process, developing good confidence interval procedures for its parameter—the probability of success p—is deceptively difficult. The binary data yield a discrete number of successes from a discrete number of trials, n. This discreteness results in actual coverage probabilities that oscillate with the n for fixed values of p (and with p for fixed n). Moreover, this oscillation necessitates a large sample size to guarantee a good coverage probability when p is close to 0 or 1.

It is well known that the Wilson procedure is superior to many existing procedures because it is less sensitive to p than any other procedures, therefore it is less costly. The procedures proposed in this article work as well as the Wilson procedure when 0.1 ≤p ≤ 0.9, and are even less sensitive (i.e., more robust) than the Wilson procedure when p is close to 0 or 1. Specifically, when the nominal coverage probability is 0.95, the Wilson procedure requires a sample size 1, 021 to guarantee that the coverage probabilities stay above 0.92 for any 0.001 ≤ min {p, 1 ?p} <0.01. By contrast, our procedures guarantee the same coverage probabilities but only need a sample size 177 without increasing either the expected interval width or the standard deviation of the interval width.     

Estimation with right‐censored observations under a semi‐Markov model

The semi‐Markov process often provides a better framework than the classical Markov process for the analysis of events with multiple states. The purpose of this paper is twofold. First, we show that in the presence of right censoring, when the right end‐point of the support of the censoring time is strictly less than the right end‐point of the support of the semi‐Markov kernel, the transition probability of the semi‐Markov process is nonidentifiable, and the estimators proposed in the literature are inconsistent in general. We derive the set of all attainable values for the transition probability based on the censored data, and we propose a nonparametric inference procedure for the transition probability using this set. Second, the conventional approach to constructing confidence bands is not applicable for the semi‐Markov kernel and the sojourn time distribution. We propose new perturbation resampling methods to construct these confidence bands. Different weights and transformations are explored in the construction. We use simulation to examine our proposals and illustrate them with hospitalization data from a recent cancer survivor study. The Canadian Journal of Statistics 41: 237–256; 2013 © 2013 Statistical Society of Canada     

Empirical likelihood for nonlinear regression models with nonignorable missing responses

This article develops three empirical likelihood (EL) approaches to estimate parameters in nonlinear regression models in the presence of nonignorable missing responses. These are based on the inverse probability weighted (IPW) method, the augmented IPW (AIPW) method and the imputation technique. A logistic regression model is adopted to specify the propensity score. Maximum likelihood estimation is used to estimate parameters in the propensity score by combining the idea of importance sampling and imputing estimating equations. Under some regularity conditions, we obtain the asymptotic properties of the maximum EL estimators of these unknown parameters. Simulation studies are conducted to investigate the finite sample performance of our proposed estimation procedures. Empirical results provide evidence that the AIPW procedure exhibits better performance than the other two procedures. Data from a survey conducted in 2002 are used to illustrate the proposed estimation procedure. The Canadian Journal of Statistics 48: 386–416; 2020 © 2020 Statistical Society of Canada     

Consistent variable selection via the optimal discovery procedure in multiple testing

In this paper, we translate variable selection for linear regression into multiple testing, and select significant variables according to testing result. New variable selection procedures are proposed based on the optimal discovery procedure (ODP) in multiple testing. Due to ODP’s optimality, if we guarantee the number of significant variables included, it will include less non significant variables than marginal p-value based methods. Consistency of our procedures is obtained in theory and simulation. Simulation results suggest that procedures based on multiple testing have improvement over procedures based on selection criteria, and our new procedures have better performance than marginal p-value based procedures.     

Tests of fit for a lognormal distribution

The problem of goodness of fit of a lognormal distribution is usually reduced to testing goodness of fit of the logarithmic data to a normal distribution. In this paper, new goodness-of-fit tests for a lognormal distribution are proposed. The new procedures make use of a characterization property of the lognormal distribution which states that the Kullback–Leibler measure of divergence between a probability density function (p.d.f) and its r-size weighted p.d.f is symmetric only for the lognormal distribution [Tzavelas G, Economou P. Characterization properties of the log-normal distribution obtained with the help of divergence measures. Stat Probab Lett. 2012;82(10):1837–1840]. A simulation study examines the performance of the new procedures in comparison with existing goodness-of-fit tests for the lognormal distribution. Finally, two well-known data sets are used to illustrate the methods developed.     

Handling missing data by deleting completely observed records

When data are missing, analyzing records that are completely observed may cause bias or inefficiency. Existing approaches in handling missing data include likelihood, imputation and inverse probability weighting. In this paper, we propose three estimators inspired by deleting some completely observed data in the regression setting. First, we generate artificial observation indicators that are independent of outcome given the observed data and draw inferences conditioning on the artificial observation indicators. Second, we propose a closely related weighting method. The proposed weighting method has more stable weights than those of the inverse probability weighting method (Zhao, L., Lipsitz, S., 1992. Designs and analysis of two-stage studies. Statistics in Medicine 11, 769–782). Third, we improve the efficiency of the proposed weighting estimator by subtracting the projection of the estimating function onto the nuisance tangent space. When data are missing completely at random, we show that the proposed estimators have asymptotic variances smaller than or equal to the variance of the estimator obtained from using completely observed records only. Asymptotic relative efficiency computation and simulation studies indicate that the proposed weighting estimators are more efficient than the inverse probability weighting estimators under wide range of practical situations especially when the missingness proportion is large.     

Testing for monotonic trend in time series based on resampling methods

In this paper, we propose several tests for monotonic trend based on the Brillinger's test statistic (1989, Biometrika, 76, 23–30). When there are highly correlated residuals or short record lengths, Brillinger's test procedure tends to have significance level much higher than the nominal level. It is found that this could be related to the discrepancy between the empirical distribution of the test statistic and the asymptotic normal distribution. Hence, in this paper, we propose three bootstrap-based procedures based on the Brillinger's test statistic to test for monotonic trend. The performance of the proposed test procedures is evaluated through an extensive Monte Carlo simulation study, and is compared to other trend test procedures in the literature. It is shown that the proposed bootstrap-based Brillinger test procedures can well control the significance levels and provide satisfactory power performance in testing the monotonic trend under different scenarios.     

A data-driven selection of the number of clusters in the Dirichlet allocation model via Bayesian mixture modelling

In this paper, we consider a Bayesian mixture model that allows us to integrate out the weights of the mixture in order to obtain a procedure in which the number of clusters is an unknown quantity. To determine clusters and estimate parameters of interest, we develop an MCMC algorithm denominated by sequential data-driven allocation sampler. In this algorithm, a single observation has a non-null probability to create a new cluster and a set of observations may create a new cluster through the split-merge movements. The split-merge movements are developed using a sequential allocation procedure based in allocation probabilities that are calculated according to the Kullback–Leibler divergence between the posterior distribution using the observations previously allocated and the posterior distribution including a ‘new’ observation. We verified the performance of the proposed algorithm on the simulated data and then we illustrate its use on three publicly available real data sets.     

Paradoxes in nonparametric tests

This paper is a continuation of one (1992) in which the author studied the paradoxes that can arise when a nonparametric statistical test is used to give an ordering of k samples and the subsets of those samples. This article characterizes the projection paradoxes that can occur when using contingency tables, complete block designs, and tests of dichotomous behaviour of several samples. This is done by examining the “dictionaries” of possible orderings of each of these procedures. Specifically, it is shown that contingency tables and complete block designs, like the Kruskal-Wallis nonparametric test on k samples, minimize the number and kinds of projection paradoxes that can occur; however, using a test of dichotomous behaviour of several samples does not. An analysis is given of two procedures used to determine the ordering of a pair of samples from a set of k samples. It is shown that these two procedures may not have anything in common.     

A comparison of the classical and the linear programming approaches to the classification problem in discriminant analysis

Several mathematical programming approaches to the classification problem in discriminant analysis have recently been introduced. This paper empirically compares these newly introduced classification techniques with Fisher's linear discriminant analysis (FLDA), quadratic discriminant analysis (QDA), logit analysis, and several rank-based procedures for a variety of symmetric and skewed distributions. The percent of correctly classified observations by each procedure in a holdout sample indicate that while under some experimental conditions the linear programming approaches compete well with the classical procedures, overall, however, their performance lags behind that of the classical procedures.     

Decompositions by sources and by subpopulations of the Pietra index: two applications to professional football teams in Italy

In this paper two innovative procedures for the decomposition of the Pietra index are proposed. The first one allows the decomposition by sources, while the second one provides the decomposition by subpopulations. As special case of the latter procedure, the “classical” decomposition in two components (within and between) can be easily obtained. A remarkable feature of both the proposed procedures is that they permit the assessment of the contribution to the Pietra index at the smallest possible level: each source for the first one and each subpopulation for the second one. To highlight the usefulness of these procedures, two applications are provided regarding Italian professional football (soccer) teams.

     

A note on the asymptotic distribution of the process capability index C pmk

Process capability indices have been widely used to evaluate the process performance to the continuous improvement of quality and productivity. The distribution of the estimator of the process capability index C _pmk is very complicated and the asymptotic distribution is proposed by Chen and Hsu [The asymptotic distribution of the processes capability index C _pmk , Comm. Statist. Theory Methods 24(5) (1995), pp. 1279–1291]. However, we found a critical error for the asymptotic distribution when the population mean is not equal to the midpoint of the specification limits. In this paper, a correct version of the asymptotic distribution is given. An asymptotic confidence interval of C _pmk by using the correct version of asymptotic distribution is proposed and the lower bound can be used to test if the process is capable. A simulation study of the coverage probability of the proposed confidence interval is shown to be satisfactory. The relation of six sigma technique and the index C _pmk is also discussed in this paper. An asymptotic testing procedure to determine if a process is capable based on the index of C _pmk is also given in this paper.     

A step-down heteroscedastic multiple comparison procedure

All-pairs power in a one-way ANOVA is the probability of detecting all true differences between pairs of means. Ramsey (1978) found that for normal distributions having equal variances, step-down multiple comparison procedures can have substantially more all-pairs power than single-step procedures, such as Tukey’s HSD, when equal sample sizes are randomly sampled from each group. This paper suggests a step-down procedure for the case of unequal variances and compares it to Dunnett's T3 technique. The new procedure is similar in spirit to one of the heteroscedastic procedures described by Hochberg and Tamhane (1987), but it has certain advantages that are discussed in the paper. Included are results on unequal sample sizes.