Oracle properties of SCAD-penalized support vector machine

In many scientific investigations, a large number of input variables are given at the early stage of modeling and identifying the variables predictive of the response is often a main purpose of such investigations. Recently, the support vector machine has become an important tool in classification problems of many fields. Several variants of the support vector machine adopting different penalties in its objective function have been proposed. This paper deals with the Fisher consistency and the oracle property of support vector machines in the setting where the dimension of inputs is fixed. First, we study the Fisher consistency of the support vector machine over the class of affine functions. It is shown that the function class for decision functions is crucial for the Fisher consistency. Second, we study the oracle property of the penalized support vector machines with the smoothly clipped absolute deviation penalty. Once we have addressed the Fisher consistency of the support vector machine over the class of affine functions, the oracle property appears to be meaningful in the context of classification. A simulation study is provided in order to show small sample properties of the penalized support vector machines with the smoothly clipped absolute deviation penalty.     

Combining Multi-class SVMs with Linear Ensemble Methods that Estimate the Class Posterior Probabilities

Roughly speaking, there is one main model of pattern recognition support vector machine, with several variants of lower popularity. On the contrary, among the different multi-class support vector machines which can be found in the literature, none is clearly favoured. On the one hand, they exhibit distinct statistical properties. On the other hand, multiple comparative studies between multi-class support vector machines and decomposition methods have highlighted the fact that each model has its advantages and drawbacks. These observations call for the evaluation of combinations of multi-class support vector machines. In this article, we study the combination of multi-class support vector machines with linear ensemble methods. Their sample complexity is low, which should prevent them from overfitting, and the outputs of two of them are estimates of the class posterior probabilities.     

Uniqueness of estimation and identifiability in mixture models

Further properties of the nonparametric maximum-likelihood estimator of a mixing distribution are obtained by exploiting the properties of totally positive kernels. Sufficient conditions for uniqueness of the estimator are given. This result is more general, and the proof is substantially simpler, than given previously. When the component density has support on N points, it is shown that all identifiable mixing distributions have support on no more than N/2 points. Identifiable mixtures are shown to lie on the boundary of the mixture model space. The maximum-likelihood estimate is shown to be unique if the vector of observations lies outside this space.     

Further results on multivariate vertical density representation and an application to random vector generation *

In this paper we further develop the theory of vertical density representation (VDR) in the multivariate case and provide a formula for the calculation of the conditional probability density of a random vector when its density value is given. An application to random vector generation is also given.     

Bayesian multivariate normal analysis under the extended reflected normal loss function

This paper considers the Bayesian analysis of the multivariate normal distribution under a new and bounded loss function, based on a reflection of the multivariate normal density function. The Bayes estimators of the mean vector can be derived for an arbitrary prior distribution of [d]. When the covariance matrix has an inverted Wishart prior density, a Bayes estimator of[d] is obtained under a bounded loss function, based on the entropy loss. Finally the admissibility of all linear estimators c[d]+ d for the mean vector is considered     

Adaptive density estimation: A curse of support?

This paper deals with the classical problem of density estimation on the real line. Most of the existing papers devoted to minimax properties assume that the support of the underlying density is bounded and known. But this assumption may be very difficult to handle in practice. In this work, we show that, exactly as a curse of dimensionality exists when the data lie in R^d, there exists a curse of support as well when the support of the density is infinite. As for the dimensionality problem where the rates of convergence deteriorate when the dimension grows, the minimax rates of convergence may deteriorate as well when the support becomes infinite. This problem is not purely theoretical since the simulations show that the support-dependent methods are really affected in practice by the size of the density support, or by the weight of the density tail. We propose a method based on a biorthogonal wavelet thresholding rule that is adaptive with respect to the nature of the support and the regularity of the signal, but that is also robust in practice to this curse of support. The threshold, that is proposed here, is very accurately calibrated so that the gap between optimal theoretical and practical tuning parameters is almost filled.     

Standardized posterior mode for the flexible use of a conjugate prior

The posterior mode under the standardized prior density is proposed to estimate a mean (vector) parameter, and its potential usefulness is discussed. Priors in this study include a conjugate prior and its generalized forms. When a prior density is factored into the standardized prior density and the supporting measure density, our suggestion is to discard the latter density and then to calculate the posterior mode of the mean under the standardized prior density. This treatment makes our choice of a prior density flexible. Implications of this treatment are discussed.     

Asymptotic properties of the maximum-likelihood estimator for a class of birth-and-death processes admitting a unique stationary distribution

The asymptotic properties of the maximum-likelihood estimator of the parameter vector for a class of birth-and-death processes admitting a unique stationary distribution are studied. Also, it is shown that identifiability of the parameter vector with respect to the likelihood implies that the Fisher information matrix is of full rank. Two special cases of biological interest are presented. One of these, the exponential birth-and-death process, is proposed as a more appropriate model of density dependence than the logistic process.     

SVM-like decision theoretical classification of high-dimensional vectors

In this paper, we consider the classification of high-dimensional vectors based on a small number of training samples from each class. The proposed method follows the Bayesian paradigm, and it is based on a small vector which can be viewed as the regression of the new observation on the space spanned by the training samples. The classification method provides posterior probabilities that the new vector belongs to each of the classes, hence it adapts naturally to any number of classes. Furthermore, we show a direct similarity between the proposed method and the multicategory linear support vector machine introduced in Lee et al. [2004. Multicategory support vector machines: theory and applications to the classification of microarray data and satellite radiance data. Journal of the American Statistical Association 99 (465), 67–81]. We compare the performance of the technique proposed in this paper with the SVM classifier using real-life military and microarray datasets. The study shows that the misclassification errors of both methods are very similar, and that the posterior probabilities assigned to each class are fairly accurate.     

A comparison of classification models to identify the Fragile X Syndrome

The main models of machine learning are briefly reviewed and considered for building a classifier to identify the Fragile X Syndrome (FXS). We have analyzed 172 patients potentially affected by FXS in Andalusia (Spain) and, by means of a DNA test, each member of the data set is known to belong to one of two classes: affected, not affected. The whole predictor set, formed by 40 variables, and a reduced set with only nine predictors significantly associated with the response are considered. Four alternative base classification models have been investigated: logistic regression, classification trees, multilayer perceptron and support vector machines. For both predictor sets, the best accuracy, considering both the mean and the standard deviation of the test error rate, is achieved by the support vector machines, confirming the increasing importance of this learning algorithm. Three ensemble methods - bagging, random forests and boosting - were also considered, amongst which the bagged versions of support vector machines stand out, especially when they are constructed with the reduced set of predictor variables. The analysis of the sensitivity, the specificity and the area under the ROC curve agrees with the main conclusions extracted from the accuracy results. All of these models can be fitted by free R programs.     

A Note on a Partition of the Likelihood Ratio Test for Autoregressive Covariance Structure

A sequence of nested hypotheses is presented for the examination of the assumption of autoregressive covariance structure in, for example, a repeated measures experiment. These hypotheses arise naturally by specifying the joint density of the underlying vector random variable as a product of conditional densities and the density of a subset of the vector random variable. The tests for all but one of the nested hypotheses are well known procedures, namely analysis of variance F-tests and Bartlett's test of equality of variances. While the procedure is based on tests of hypotheses, it may be viewed as an exploratory tool which can lead to model identification. An example is presented to illustrate the method.     

Adaptive kernel scaling support vector machine with application to a prostate cancer image study

The support vector machine (SVM) is a popularly used classifier in applications such as pattern recognition, texture mining and image retrieval owing to its flexibility and interpretability. However, its performance deteriorates when the response classes are imbalanced. To enhance the performance of the support vector machine classifier in the imbalanced cases we investigate a new two stage method by adaptively scaling the kernel function. Based on the information obtained from the standard SVM in the first stage, we conformally rescale the kernel function in a data adaptive fashion in the second stage so that the separation between two classes can be effectively enlarged with incorporation of observation imbalance. The proposed method takes into account the location of the support vectors in the feature space, therefore is especially appealing when the response classes are imbalanced. The resulting algorithm can efficiently improve the classification accuracy, which is confirmed by intensive numerical studies as well as a real prostate cancer imaging data application.     

On expected volumes of multidimensional confidence sets associated with the usual and adjusted likelihoods

We consider a general multiparameter set-up, where both the interest and the nuisance parameters are possibly vector valued. We derive an explicit higher order asymptotic formula to compare the expected volumes of confidence sets given by likelihood ratio statistics arising from the usual profile likelihood and various adjustments thereof. Our general framework also allows us to include highest posterior density regions, with approximate frequentist validity, in the study. The fact that our interest parameter is possibly vector valued complicates the derivation and warrants the development of special tools and techniques.     

Generation of multivariate distributions by vertical density representation

Troutt (1991,1993) proposed the idea of the vertical density representation (VDR) based on Box-Millar method. Kotz, Fang and Liang (1997) provided a systematic study on the multivariate vertical density representation (MVDR). Suppose that we want to generate a random vector X[d]Rⁿthat has a density function ?(x). The key point of using the MVDR is to generate the uniform distribution on [D]_?(v) = {x :?(x) = v} for any v > 0 which is the surface in RⁿIn this paper we use the conditional distribution method to generate the uniform distribution on a domain or on some surface and based on it we proposed an alternative version of the MVDR(type 2 MVDR), by which one can transfer the problem of generating a random vector X with given density f to one of generating (X, X_n+i) that follows the uniform distribution on a region in Rⁿ⁺¹defined by ?. Several examples indicate that the proposed method is quite practical.     

Simulating from Exchangeable Archimedean Copulas

This article considers identifying the existence of the local likelihood estimator in binary regression. A simple method for the identification is proposed, which is derived by recognizing the problem as a linear complementarity problem through a support vector machine problem with soft margins.     

The superharmonic condition for simultaneous estimation of means in exponential familles

The mean vector associated with several independent variates from the exponential subclass of Hudson (1978) is estimated under weighted squared error loss. In particular, the formal Bayes and “Stein-like” estimators of the mean vector are given. Conditions are also given under which these estimators dominate any of the “natural estimators”. Our conditions for dominance are motivated by a result of Stein (1981), who treated the N_p (θ, I) case with p ≥ 3. Stein showed that formal Bayes estimators dominate the usual estimator if the marginal density of the data is superharmonic. Our present exponential class generalization entails an elliptic differential inequality in some natural variables. Actually, we assume that each component of the data vector has a probability density function which satisfies a certain differential equation. While the densities of Hudson (1978) are particular solutions of this equation, other solutions are not of the exponential class if certain parameters are unknown. Our approach allows for the possibility of extending the parametric Stein-theory to useful nonexponential cases, but the problem of nuisance parameters is not treated here.     

Robust Bayesian analysis of the linear regression model

The robust Bayesian analysis of the linear regression model is presented under the assumption of a mixture of g-prior distributions for the parameters and ML-II posterior density for the coefficient vector is derived. Robustness properties of the ML-II posterior mean are studied. Utilizing the ML-II posterior density, robust Bayes predictors for the future values of the dependent variable are also obtained.     

工业取用水监测奇异数据挖掘与重构方法

提高工业取用水监测数据质量是目前国家水资源监控能力建设的重要内容,而奇异值问题已成为影响监测数据质量的关键短板。本文在解析现阶段工业取用水监测数据奇异值主要类型基础上,以国家水资源管理系统数据库中工业取用水监测数据为样本,利用小波变换模极大值模型提取工业取用水监测数据时频变化特征,并利用傅里叶函数对其残差序列进行修正,进而运用相对误差控制方法挖掘监测数据奇异值。在此基础上,采用混沌粒子群优化的最小二乘支持向量机模型重构填补奇异值数据。研究结果表明：小波变换模极大值模型能够较好地提取工业取用水监测数据序列的时频变化特征,但是同时容易导致监测数据的信息损失,利用傅里叶函数对小波变换进行残差修正则可进一步提升取用水监测数据序列的特征提取效果;以小波变换模极大值特征序列为基础,通过相对误差控制可实现对监测数据奇异值的高效挖掘;对于挖掘出的奇异值重构填补问题,可选取混沌粒子群优化的最小二乘支持向量机模型,其重构精度要优于多项式曲线拟合等传统统计学方法和普通最小二乘支持向量机模型。上述工业取用水监测数据奇异值挖掘重构策略为现阶段国家水资源监控能力建设的推进提供了重要技术方法支持。     

Generating random variates from a bicompositional Dirichlet distribution

A composition is a vector of positive components summing to a constant. The sample space of a composition is the simplex, and the sample space of two compositions, a bicomposition, is a Cartesian product of two simplices. We present a way of generating random variates from a bicompositional Dirichlet distribution defined on the Cartesian product of two simplices using the rejection method. We derive a general solution for finding a dominating density function and a rejection constant and also compare this solution to using a uniform dominating density function. Finally, some examples of generated bicompositional random variates, with varying number of components, are presented.