Logistic模型对非平衡数据的敏感性:测度、修正与比较

本文以UCI数据库为研究样本,分析logistic模型对不同程度非平衡数据的敏感性.研究表明:①数据非平衡程度越高,logistic回归对稀有类的识别能力越差.②相对于其他修正方法,OSS方法的改进效果不显著且不稳定;相对于复杂抽样,简单抽样修正结果更优.③AUC值不适宜于非平衡数据条件下的模型选择,因为在非平衡数据条件下,它不能有效区分四种修正方法的优劣,而且修正前后的差异亦不能辨.     

一种加权的MCLP分类模型及其在不平衡数据集上的应用

在数据挖掘的分类问题中，经常出现数据集内类别不平衡现象。大部分分类方法对于不平衡数据集内的小类数据，分类精度并不理想。文章分析了多目标线性规划分类方法（简称MCLP）在不平衡数据集上的表现；然后从模型角度，提出了面向不平衡数据集的加权MCLP分类模型。从理论上分析了加权MCLP分类模型的有效性，并从实证角度，与其他方法进行了比较。     

基于自然最近邻的不平衡数据欠采样方法

针对不平衡数据集中的少数类样本在实际应用中分类准确率较低的问题，提出一种利用多数类样本的自然最近邻进行欠采样的数据处理方法。自然最近邻算法根据每个样本的分布特征动态地为样本选择数量不同的自然最近邻样本，通过自然最近邻的个数反映样本分布的疏密程度。文章所提方法先计算多数类样本在整体数据集中的自然最近邻，根据自然最近邻情况移除多数类中的噪声样本和局部密度较小的样本，再计算剩余样本的相似度，保留密集区域中的代表性样本，去掉部分冗余样本，获得平衡数据集。该方法的计算无须预先指定参数，减少了欠采样过程中多数类分类信息的损失。对比实验利用支持向量机对不同欠采样方法平衡后的12个数据集进行分类，结果表明此方法在大多数数据集上具有较优的分类性能，提升了少数类样本的分类准确率。     

基于半静态分层抽样的模糊聚类分析方法的改进

基于统计模型的模糊聚类算法的时间复杂度在数据集规模超过一定数量级时是计算不可行的,解决时间复杂度的一个行之有效的方法是抽样.文章通过对静态抽样进行改进,设计了一种半静态抽样法,使样本数据集最大程度得保持原数据集的信息,并保证聚类结果的不失真性;最后通过实证分析,比较并证明了该方法是有效的.     

基于非负矩阵分解的函数型聚类算法

函数型聚类分析算法涉及投影和聚类两个基本要素。通常,最优投影结果未必能够有效地保留类别信息,从而影响后续聚类效果。为此,本文梳理了函数型聚类的构成要素及运行过程;借助非负矩阵分解的聚类特性,提出了基于非负矩阵分解的函数型聚类算法,构建了“投影与聚类”并行的实现框架,并采用交替迭代方法更新求解,分析了算法的计算时间复杂度。针对随机模拟数据验证和语音识别数据的实例检验结果显示,该函数型聚类算法有助于提高聚类效果;针对北京市二氧化氮(NO2)污染物小时浓度数据的实例应用表明,该函数型聚类算法对空气质量监测点类型的区分能够充分识别站点布局的空间模式,具有良好的实际应用价值。     

对丢失目标总体单元抽样诓误差一种补救方法的探讨

一、抽样框误差及其分类 抽样调查的误差包括抽样误差和非抽样误差两类.对于非抽样误差,可以分为抽样框误差、无回答误差和计量误差三类.抽样框误差是由不完善的抽样框引起的误差;无回答误差是由于种种原因没有能够对被抽出的样本单元进行计量,没有获得有关这些单元的数据而引起的误差;计量误差是由调查所获得的数据与调查项目的真值不一致而产生的误差.     

两分类不平衡数据的Boosting算法

Boosting算法是一类串行的集成算法,可用于分类和回归。不同的算法由不同的损失与不同的集成方式构成。文章提出了一种自适应地处理分类中的不平衡数据的Boosting算法Baboost。实验证明该算法能有效地减小各个类内部的预测误差。     

一种基于改进NISD的偏二叉树马田系统的数据多分类算法

为解决马田系统多分类算法存在的样本重复训练以及分类准确率下降等问题,文章提出了一种基于改进的类间相似方向数（Number of Inter-class Similarity Direction,NISD）的偏二叉树马田系统多分类算法。该算法利用马氏距离改进类间相似方向数,获得更为科学的样本分类顺序,依此顺序自上而下生成整个偏二叉树,在非叶子节点构造马田系统二分类器,生成最终的分类模型。对于含k个类别的待分类样本,该算法只用训练k-1个二分类器,便可得到马田系统多分类模型,与此同时,层层剥离样本减少了样本的重复训练。UCI数据集实验结果表明,该算法分类效率更高,分类准确率也较高。     

基于BT-SVM模型组合的动态加权多分类算法研究

在AI领域中,备受关注的一个问题是如何获得更好的分类,尤其是对于多分类的情形。目前,针对多分类算法已取得了大量的研究成果,很多较为高效的多分类算法也已应用到实践中,而对于多分类算法的研究仍然备受关注。以BT-SVM为基分类器,提出一种带阈值的新型动态加权多分类器集成的方法,并通过模拟和实证分析验证该算法的有效性,研究表明该算法对于平衡和非平衡数据的分类效果表现得都比较优良。     

一种基于聚类与离散化算法的分类方法

在采用聚类方法产生训练集的基础上,运用粗集理论离散化预处理该训练集,可以更好的提高分类精度.文章运用PAM算法聚类原始样本构成训练集,再利用布尔逻辑和粗集理论结合的离散化算法离散化该训练集,并以此离散化的训练集训练分类器.实验结果证明,基于该方法在相同的数据集上分类,比仅基于PAM算法预处理的RDDTE方法产生的分类精度最高提高了15.5%,且选用更少量的训练集.     

Dealing with Imbalanced Dataset: A Re-sampling Method Based on the Improved SMOTE Algorithm

Most classification models have presented an imbalanced learning state when dealing with the imbalanced datasets. This article proposes a novel approach for learning from imbalanced datasets, which based on an improved SMOTE (synthetic Minority Over-sampling technique) algorithm. By organically combining the over-sampling and the under-sampling method, this approach aims to choose neighbors targetedly and synthesize samples with different strategy. Experiments show that most classifiers have achieved an ideal performance on the classification problem of the positive and negative class after dealing imbalanced datasets with our algorithm.     

General mixed Poisson regression models with varying dispersion

A general class of mixed Poisson regression models is introduced. This class is based on a mixing between the Poisson distribution and a distribution belonging to the exponential family. With this, we unified some overdispersed models which have been studied separately, such as negative binomial and Poisson inverse gaussian models. We consider a regression structure for both the mean and dispersion parameters of the mixed Poisson models, thus extending, and in some cases correcting, some previous models considered in the literature. An expectation–maximization (EM) algorithm is proposed for estimation of the parameters and some diagnostic measures, based on the EM algorithm, are considered. We also obtain an explicit expression for the observed information matrix. An empirical illustration is presented in order to show the performance of our class of mixed Poisson models. This paper contains a Supplementary Material.     

Finite mixtures of multivariate Poisson distributions with application

In the present paper we examine finite mixtures of multivariate Poisson distributions as an alternative class of models for multivariate count data. The proposed models allow for both overdispersion in the marginal distributions and negative correlation, while they are computationally tractable using standard ideas from finite mixture modelling. An EM type algorithm for maximum likelihood (ML) estimation of the parameters is developed. The identifiability of this class of mixtures is proved. Properties of ML estimators are derived. A real data application concerning model based clustering for multivariate count data related to different types of crime is presented to illustrate the practical potential of the proposed class of models.     

改进BP神经网络在经济预测中的应用

针对传统BP学习算法收敛速度慢、对步长依赖明显等缺点，提出一种利用搜索较优步长的BP算法。其在网络训练中，能够在每次迭代中搜索出一个相对合理的步长，从而使步长的选择对学习速度的影响大大降低。对经济预测仿真结果表明，新算法对步长选择的依赖性小于传统BP算法。     

Interacting multiple try algorithms with different proposal distributions

We introduce a new class of interacting Markov chain Monte Carlo (MCMC) algorithms which is designed to increase the efficiency of a modified multiple-try Metropolis (MTM) sampler. The extension with respect to the existing MCMC literature is twofold. First, the sampler proposed extends the basic MTM algorithm by allowing for different proposal distributions in the multiple-try generation step. Second, we exploit the different proposal distributions to naturally introduce an interacting MTM mechanism (IMTM) that expands the class of population Monte Carlo methods and builds connections with the rapidly expanding world of adaptive MCMC. We show the validity of the algorithm and discuss the choice of the selection weights and of the different proposals. The numerical studies show that the interaction mechanism allows the IMTM to efficiently explore the state space leading to higher efficiency than other competing algorithms.     

Genetic Algorithm in the Wavelet Domain for Large p Small n Regression

Many areas of statistical modeling are plagued by the “curse of dimensionality,” in which there are more variables than observations. This is especially true when developing functional regression models where the independent dataset is some type of spectral decomposition, such as data from near-infrared spectroscopy. While we could develop a very complex model by simply taking enough samples (such that n > p), this could prove impossible or prohibitively expensive. In addition, a regression model developed like this could turn out to be highly inefficient, as spectral data usually exhibit high multicollinearity. In this article, we propose a two-part algorithm for selecting an effective and efficient functional regression model. Our algorithm begins by evaluating a subset of discrete wavelet transformations, allowing for variation in both wavelet and filter number. Next, we perform an intermediate processing step to remove variables with low correlation to the response data. Finally, we use the genetic algorithm to perform a stochastic search through the subset regression model space, driven by an information-theoretic objective function. We allow our algorithm to develop the regression model for each response variable independently, so as to optimally model each variable. We demonstrate our method on the familiar biscuit dough dataset, which has been used in a similar context by several researchers. Our results demonstrate both the flexibility and the power of our algorithm. For each response variable, a different subset model is selected, and different wavelet transformations are used. The models developed by our algorithm show an improvement, as measured by lower mean error, over results in the published literature.     

Comparative analysis of modern optimization tools for the p-median problem

This paper develops a study on different modern optimization techniques to solve the p-median problem. We analyze the behavior of a class of evolutionary algorithm (EA) known as cellular EA (cEA), and compare it against a tailored neural network model and against a canonical genetic algorithm for optimization of the p-median problem. We also compare against existing approaches including variable neighborhood search and parallel scatter search, and show their relative performances on a large set of problem instances. Our conclusions state the advantages of using a cEA: wide applicability, low implementation effort and high accuracy. In addition, the neural network model shows up as being the more accurate tool at the price of a narrow applicability and larger customization effort.     

Weighted Support Vector Machine Using k-Means Clustering

The support vector machine (SVM) has been successfully applied to various classification areas with great flexibility and a high level of classification accuracy. However, the SVM is not suitable for the classification of large or imbalanced datasets because of significant computational problems and a classification bias toward the dominant class. The SVM combined with the k-means clustering (KM-SVM) is a fast algorithm developed to accelerate both the training and the prediction of SVM classifiers by using the cluster centers obtained from the k-means clustering. In the KM-SVM algorithm, however, the penalty of misclassification is treated equally for each cluster center even though the contributions of different cluster centers to the classification can be different. In order to improve classification accuracy, we propose the WKM–SVM algorithm which imposes different penalties for the misclassification of cluster centers by using the number of data points within each cluster as a weight. As an extension of the WKM–SVM, the recovery process based on WKM–SVM is suggested to incorporate the information near the optimal boundary. Furthermore, the proposed WKM–SVM can be successfully applied to imbalanced datasets with an appropriate weighting strategy. Experiments show the effectiveness of our proposed methods.     

Adaptive grid semidefinite programming for finding optimal designs

We find optimal designs for linear models using a novel algorithm that iteratively combines a semidefinite programming (SDP) approach with adaptive grid techniques. The proposed algorithm is also adapted to find locally optimal designs for nonlinear models. The search space is first discretized, and SDP is applied to find the optimal design based on the initial grid. The points in the next grid set are points that maximize the dispersion function of the SDP-generated optimal design using nonlinear programming. The procedure is repeated until a user-specified stopping rule is reached. The proposed algorithm is broadly applicable, and we demonstrate its flexibility using (i) models with one or more variables and (ii) differentiable design criteria, such as A-, D-optimality, and non-differentiable criterion like E-optimality, including the mathematically more challenging case when the minimum eigenvalue of the information matrix of the optimal design has geometric multiplicity larger than 1. Our algorithm is computationally efficient because it is based on mathematical programming tools and so optimality is assured at each stage; it also exploits the convexity of the problems whenever possible. Using several linear and nonlinear models with one or more factors, we show the proposed algorithm can efficiently find optimal designs.     

Inferring large graphs using $$\ell _1$$-penalized likelihood

We address the issue of recovering the structure of large sparse directed acyclic graphs from noisy observations of the system. We propose a novel procedure based on a specific formulation of the \(\ell _1\)-norm regularized maximum likelihood, which decomposes the graph estimation into two optimization sub-problems: topological structure and node order learning. We provide convergence inequalities for the graph estimator, as well as an algorithm to solve the induced optimization problem, in the form of a convex program embedded in a genetic algorithm. We apply our method to various data sets (including data from the DREAM4 challenge) and show that it compares favorably to state-of-the-art methods. This algorithm is available on CRAN as the R package GADAG.