函数型数据的共同主成分分析探究及展望

函数型数据的主成分分析(FPCA)已经成功应用在许多领域,但它主要研究的是单样本问题.本文详细讨论了一种新近发展的函数型数据分析的理论--函数型共同主成分(CGPC)分析方法,它主要应用于检验两组函数型随机样本的分布情况.CFPC方法的理论基础是将两组函数型样本进行Karhunen-Loeve(KL)展开,并用Bootstrap方法检验两组样本的均值函数、特征值和特征函数的一致性.最后,我们对CFPC的理论研究和应用前景进行了展望.     

基于Kolmogorov-Smirnov距离的统计诊断

统计诊断是数据分析的重要组成部分,其主要任务是检测已知观测数据在用既定模型拟合时的合理性.文章利用Kolmogorov-Smimov距离(K-S距离)对残差的经验分布函数进行了分析,进而研究了异常点的统计诊断问题;最后利用Forbes数据说明了该方法的有效性.     

基于部分函数型线性回归模型的改进

金融市场的交易是不间断的,价格始终高频的更新,在金融数据的研究中,经常遇到函数型数据.文章主要建立部分函数型线性回归模型,分析函数型数据在上证指数预测中的应用,根据函数型数据分析的原理及其求解主成分分析的方法,使用Matlab对上证指数进行预测.     

函数性数据的统计分析：思想、方法和应用

 摘　　要:实际中，越来越多的研究领域所收集到的样本观测数据具有函数性特征，这种函数性数据是融合时间序列和横截面两者的数据，有些甚是曲线或其他函数图像。虽然计量经济学近二十多年来发展的面板数据分析方法，具有很好的应用价值，但是面板数据只是函数性数据的一种特殊类型，且其分析方法太过于依赖模型的线性结构和假设条件等。本文基于函数性数据的普遍特征，介绍一种对其进行分析的全新方法，并率先使用该方法对经济函数性数据进行分析，拓展了函数性数据分析的应用范围。分析结果表明，函数性数据分析方法，较之计量经济学和其他统计方法具有更多的优越性，尤其能够揭示其他方法所不能揭示的数据特征     

出现较多DEA(弱)有效单元的被评价对象的排序问题

一、引言 DEA(Data Envelopment Analysis)方法即数据包络分析方法,是对多指标投入和多指标产出的相同类型部门,进行相对有效性综合评价的一种新方法,也是研究多投入多产出生产函数的有力工具;DEA方法以相对效率概念为基础,自1978年A.charnes,w.w.cooper和E.Rhodes提出第一个DEA模型模型以来,又陆续出现了C2GS2,BC2,FG和ST以及考虑带有锥结构的综合DEA模型,至今已形成了关于效率、生产可能集、生产前沿面等概念的完整的理论、方法和模型的DEA研究领域.     

大数据与传统统计分析

1 大数据的研究背景 2008年,Nature发表专刊“Big Data”,同年,Science杂志发表文章“Big Data:Science in the Petabyte Era”,大数据开始广泛传播.2012年2月,格雷布林克(Grobelink.M)在《纽约时报》的一篇专栏中称,“大数据时代”已经降临,在商业、经济及其他领域中,管理者决策越来越依靠数据分析,而不是依靠经验和直觉. 大数据指的是无法在可容忍的时间内,使用传统IT技术和软硬件工具,对其进行处理和服务的数据集.鲍里斯·埃韦尔松2012年将大数据的特点总结为4个V,即Volume(大量性)、Variety(多样性)、Velocity(高速性)和Variability(易变性).     

纵向部分线性变系数EV模型的估计

纵向数据是随着时间变化对个体进行重复观测而得到的一种相关性数据,广泛出现在诸多科学研究领域。在对个体进行观测时,测量误差不可避免,忽略测量误差往往会导致有偏估计。本文利用二次推断函数方法研究关于纵向数据的参数部分和非参数部分协变量均含有测量误差的部分线性变系数测量误差(errors-in-variables, EV)模型的估计问题。利用B样条逼近模型中的未知系数函数,构造关于回归参数和B样条系数的偏差修正的二次推断函数以处理个体内相关性和测量误差,得到回归参数和变系数的偏差修正的二次推断函数估计,然后证明了估计方法和结果的渐近性质。数值模拟和实例数据分析结果显示本文提出的方法具有一定的实用价值。     

基于函数性视角的经济数据分析——以主微分分析方法为例

基于经济数据的函数性特征,引入函数型数据分析方法,研究发现经济数据中的面板数据可作为函数型数据的特例,函数型数据分析方法在处理高维数据、缺失数据以及样本观测点不规则分布等特殊的数据类型有独特的优势。着重介绍和拓展了主微分分析方法,在集合了主成分分析方法优势的同时从微分方程的解出发探讨数据的特征。通过对全国银行间同业拆借利率进行主微分分析,显示出主微分分析方法能够揭示其它方法所不能反映的数据特征。     

基于基函数展开的函数型数据聚类方法

文章在一个一般性的框架下研究了利用基函数展开进行函数型数据聚类的问题.在这个框架之下,大量传统的聚类方法都可以直接应用到函数型数据分析.另外,我们将Pearson相似系数引入函数型数据聚类分析,解决了欧式距离无法刻画曲线之间形态差异的问题.     

微观计量经济学最新进展

一、前言世纪之交的诺贝尔经济学奖授予了两位美国经济学家詹姆士·Ｊ·赫克曼 (ＪａｍｅｓＪ Ｈｅｃｋｍａｎ)教授和丹尼尔·Ｌ·麦克法登 (ＤａｎｉｅＩＬ ＭｃＦａｄｄｅｎ)教授 ,以表彰他们对微观计量经济学所做出的杰出贡献 ,前者的贡献“是发展了选择性样本数据分析的理论和方法” ,后者的贡献“是发展了对自选择行为进行分析的理论和方法”。微观计量经济学是介于经济学和统计学之间的边缘科学 ,它是研究微观数据———即大量个人、家庭或企业的经济信息的经济理论和统计方法。微观数据可以是某一时点的横向数据 ,也可以是相同观测对…     

Two Samples Tests for Functional Data

Data in many experiments arises as curves and therefore it is natural to use a curve as a basic unit in the analysis, which is in terms of functional data analysis (FDA). Functional curves are encountered when units are observed over time. Although the whole function curve itself is not observed, a sufficiently large number of evaluations, as is common with modern recording equipment, is assumed to be available. In this article, we consider the statistical inference for the mean functions in the two samples problem drawn from functional data sets, in which we assume that functional curves are observed, that is, we consider the test if these two groups of curves have the same mean functional curve when the two groups of curves without noise are observed. The L ²-norm based and bootstrap-based test statistics are proposed. It is shown that the proposed methodology is flexible. Simulation study and real-data examples are used to illustrate our techniques.     

A general methodology for bootstrapping in non-parametric frontier models

The Data Envelopment Analysis method has been extensively used in the literature to provide measures of firms' technical efficiency. These measures allow rankings of firms by their apparent performance. The underlying frontier model is non-parametric since no particular functional form is assumed for the frontier model. Since the observations result from some data-generating process, the statistical properties of the estimated efficiency measures are essential for their interpretations. In the general multi-output multi-input framework, the bootstrap seems to offer the only means of inferring these properties (i.e. to estimate the bias and variance, and to construct confidence intervals). This paper proposes a general methodology for bootstrapping in frontier models, extending the more restrictive method proposed in Simar & Wilson (1998) by allowing for heterogeneity in the structure of efficiency. A numerical illustration with real data is provided to illustrate the methodology.     

A Comparison of Hierarchical Methods for Clustering Functional Data

Functional data analysis (FDA)—the analysis of data that can be considered a set of observed continuous functions—is an increasingly common class of statistical analysis. One of the most widely used FDA methods is the cluster analysis of functional data; however, little work has been done to compare the performance of clustering methods on functional data. In this article, a simulation study compares the performance of four major hierarchical methods for clustering functional data. The simulated data varied in three ways: the nature of the signal functions (periodic, non periodic, or mixed), the amount of noise added to the signal functions, and the pattern of the true cluster sizes. The Rand index was used to compare the performance of each clustering method. As a secondary goal, clustering methods were also compared when the number of clusters has been misspecified. To illustrate the results, a real set of functional data was clustered where the true clustering structure is believed to be known. Comparing the clustering methods for the real data set confirmed the findings of the simulation. This study yields concrete suggestions to future researchers to determine the best method for clustering their functional data.     

Applying functional data analysis to assess tele-interpersonal psychotherapy's efficacy to reduce depression

The use of parametric linear mixed models and generalized linear mixed models to analyze longitudinal data collected during randomized control trials (RCT) is conventional. The application of these methods, however, is restricted due to various assumptions required by these models. When the number of observations per subject is sufficiently large, and individual trajectories are noisy, functional data analysis (FDA) methods serve as an alternative to parametric longitudinal data analysis techniques. However, the use of FDA in RCTs is rare. In this paper, the effectiveness of FDA and linear mixed models (LMMs) was compared by analyzing data from rural persons living with HIV and comorbid depression enrolled in a depression treatment randomized clinical trial. Interactive voice response systems were used for weekly administrations of the 10-item Self-Administered Depression Scale (SADS) over 41 weeks. Functional principal component analysis and functional regression analysis methods detected a statistically significant difference in SADS between telphone-administered interpersonal psychotherapy (tele-IPT) and controls but linear mixed effects model results did not. Additional simulation studies were conducted to compare FDA and LMMs under a different nonlinear trajectory assumption. In this clinical trial with sufficient per subject measured outcomes and individual trajectories that are noisy and nonlinear, we found FDA methods to be a better alternative to LMMs.     

Multilayer Perceptron with Functional Inputs: an Inverse Regression Approach

Abstract.  Functional data analysis is a growing research field as more and more practical applications involve functional data. In this paper, we focus on the problem of regression and classification with functional predictors: the model suggested combines an efficient dimension reduction procedure [functional sliced inverse regression, first introduced by Ferré & Yao ( Statistics , 37, 2003 , 475)], for which we give a regularized version, with the accuracy of a neural network. Some consistency results are given and the method is successfully confronted to real-life data.     

大数据时代统计学的重构与创新

本文集中介绍了多位顶尖统计学家在大数据研究方面的新进展,内容涉及大数据背景下政府统计需求,统计设计,统计学理论框架的重构,统计学利用大数据在基因学、天文学、宇宙学、流行病学、经济金融学、生命科学和工程学等领域中的应用,以及大数据人才培养问题等。     

Functional Modelling and Classification of Longitudinal Data*

Abstract. We review and extend some statistical tools that have proved useful for analysing functional data. Functional data analysis primarily is designed for the analysis of random trajectories and infinite‐dimensional data, and there exists a need for the development of adequate statistical estimation and inference techniques. While this field is in flux, some methods have proven useful. These include warping methods, functional principal component analysis, and conditioning under Gaussian assumptions for the case of sparse data. The latter is a recent development that may provide a bridge between functional and more classical longitudinal data analysis. Besides presenting a brief review of functional principal components and functional regression, we develop some concepts for estimating functional principal component scores in the sparse situation. An extension of the so‐called generalized functional linear model to the case of sparse longitudinal predictors is proposed. This extension includes functional binary regression models for longitudinal data and is illustrated with data on primary biliary cirrhosis.     

Multivariate analysis of variance for functional data

Functional data are being observed frequently in many scientific fields, and therefore most of the standard statistical methods are being adapted for functional data. The multivariate analysis of variance problem for functional data is considered. It seems to be of practical interest similarly as the one-way analysis of variance for such data. For the MANOVA problem for multivariate functional data, we propose permutation tests based on a basis function representation and tests based on random projections. Their performance is examined in comprehensive simulation studies, which provide an idea of the size control and power of the tests and identify differences between them. The simulation experiments are based on artificial data and real labeled multivariate time series data found in the literature. The results suggest that the studied testing procedures can detect small differences between vectors of curves even with small sample sizes. Illustrative real data examples of the use of the proposed testing procedures in practice are also presented.     

A functional data approach to model score difference process in professional basketball games

In this paper, we investigate the progress of score difference (between home and away teams) in professional basketball games employing functional data analysis (FDA). The observed score difference is viewed as the realization of the latent intensity process, which is assumed to be continuous. There are two major advantages of modeling the latent score difference intensity process using FDA: (1) it allows for arbitrary dependent structure among score change increments. This removes potential model mis-specifications and accommodates momentum which is often observed in sports games. (2) further statistical inferences using FDA estimates will not suffer from inconsistency due to the issue of having a continuous model yet discretely sampled data. Based on the FDA estimates, we define and numerically characterize momentum in basketball games and demonstrate its importance in predicting game outcomes.