金融高维协方差阵的估计及其应用

文章克服了传统高维协方差阵估计方法的缺点,将主成分和门限方法相结合,提出了门限主成分正交补(TPO)估计量,该估计量主要通过前K个主成分来刻画高维协方差阵的信息,通过引入合适的门限函数来对矩阵的正交补进行稀疏估计,从而有效的降低了数据的维度并剔除了噪声的影响.模拟和实证研究发现:较严格的因子(SFM)模型而言,门限主成分正交补(TPO)模型明显提高了协方差阵的估计效率,并且将其应用在投资组合时,投资者获得了更高的收益和经济福利.     

基于高维数据的改进CCC-GARCH模型的估计及应用

高维数据给传统的协方差阵估计方法带来了巨大的挑战,数据维度和噪声的影响使传统的CCCGARCH模型估计起来较为困难。将主成分和门限方法有效结合,应用到CCC-GARCH模型的估计中,提出基于主成分正交补门限方法的CCC-GARCH模型(PTCCC-GARCH)。PTCCC模型主要通过前K个最优主成分来刻画大维协方差阵的信息,并通过门限函数以剔除噪声的影响。通过模拟和实证研究发现:较CCCGARCH模型而言,PTCCC-GARCH模型明显提高了高维协方差阵的估计和预测效率;并且将其应用在投资组合时,投资者获得了更高的投资收益和经济福利。     

高维厚尾金融数据协方差阵的统计估计及应用

近年来,关于高维协方差阵估计的研究大多是在正态分布的假定下进行的,少有研究考虑金融数据的厚尾特征对协方差阵估计的影响。在提出新方法估计厚尾金融海量数据协方差阵的基础上,先引入乔列斯基分解法,将复杂的协方差阵估计问题转化为一系列的回归模型;再在回归模型的估计过程中引入RA-Lasso方法,使其在解决维数诅咒的同时,还考虑由于数据的厚尾特征而引起的估计偏差问题;通过模拟和实证研究发现,新的方法明显提高了协方差阵的估计效率,并且使投资者获得了更高的收益。     

析主成分分析和因子分析——以Frobenius范数下矩阵近似的新视角

本文通过引入数据阵在Frobenius范数下的最优近似等概念来重新探讨主成分和因子分析。我发现，主成分分析中主成分和因子分析中因子得分（通过主成分解因子载荷，然后用最小二乘解因子得分）的估计为数据阵的最优近似（在Frobenius范数下）在不同正交坐标方向矩阵下的坐标。两种方法分别采用了不同的约束条件分解的最优近似（在Frobenius范数下），因为该分解并不唯一。     

基于动态收缩法的大维协方差阵的估计及其应用

文章将单因子协方差阵和样本协方差阵相结合,通过对它们进行最优加权平均,提出了新的协方差阵估计方法——动态加权收缩估计量(DWS).该估计量一方面通过选择最优的权重来平衡协方差阵估计的偏差和误差;另一方面估计的是大维数据的动态协方差阵,在估计过程中考虑了前期信息的影响.通过模拟和实证研究发现:较传统的协方差阵估计方法而言,DWS估计量明显提高了大维协方差阵的估计效率;并且将其应用在投资组合时,投资者获得了更高的收益和经济福利.     

高维GARCH型数据控制图的主成分方法

高维GARCH模型逐渐在金融市场中建立并使用,而高维控制图应用较少,文章首次采用主成分的方法建立高维GARCH控制图,能够有效改善控制图不易识别和保存数据信息量等问题,以美元汇率和股票市场2008-2009年共262个数据为例,建立汇率市场与股票市场的合成控制图,实证表明该控制图能够准确、有效识别异常点,起到监控和预警的作用。     

稳健高效的高维成分数据近似零值插补方法及应用

随着计算机技术的迅猛发展,高维成分数据不断涌现并伴有大量近似零值和缺失,数据的高维特性不仅给传统统计方法带来了巨大的挑战,其厚尾特征、复杂的协方差结构也使得理论分析难上加难。于是如何对高维成分数据的近似零值进行稳健的插补,挖掘潜在的内蕴结构成为当今学者研究的焦点。对此,本文结合修正的EM算法,提出基于R型聚类的Lasso-分位回归插补法(SubLQR)对高维成分数据的近似零值问题予以解决。与现有高维近似零值插补方法相比,本文所提出的SubLQR具有如下优势。①稳健全面性:利用Lasso-分位回归方法,不仅可以有效地探测到响应变量的整个条件分布,还能提供更加真实的高维稀疏模式;②有效准确性:采用基于R型聚类的思想进行插补,可以降低计算复杂度,极大提高插补的精度。模拟研究证实,本文提出的SubLQR高效灵活准确,特别在零值、异常值较多的情形更具优势。最后将SubLQR方法应用于罕见病代谢组学研究中,进一步表明本文所提出的方法具有广泛的适用性。     

岭型主成分估计下数据删除模型的强影响分析

文章讨论了在岭型主成分估计下的数据删除模型,得到了该模型与最小二乘估计下模型的诊断统计量之间的等价关系,在此基础上根据W-K统计量的思想提出了两种度量方法,并通过实例论证了该方法的可行性。     

多重稳健的高维缺失数据插补研究

缺失数据问题在抽样调查、社会科学、流行病等领域普遍存在，这一现象在高维情形下更为凸显；而与高维数据相伴的信息海量化、复杂化、异质化、缺失化等问题，给高维缺失数据理论建立及应用研究带来极大的挑战。如何建立一种稳健高效的高维缺失数据插补方法，已成为当今学者研究的焦点。为解决上述难题，创新性地将增强的逆概率加权(IPW)与加法模型融合，应用协变量平衡倾向评分法(CBPS)估计缺失概率，提出一种适用于高维缺失数据的可加协变量平衡倾向评分插补方法(CBPS-AM),期望对高维缺失问题提供更为有效的解决方案。CBPS-AM方法不仅具有多重稳健性，避免了模型误设带来的严重风险，还能够有效规避高维缺失数据具有厚尾分布而使得传统插补方法失效的问题，起到双重降维的作用，实现建模的灵活性与广泛适用性。其次借鉴广义矩估计方法和Backfitting算法给出了CBPS估计算法，该算法简洁有效，能够提高数据使用效率与插补精度，同时研究了估计量的理论性质，对比了所提方法与传统方法在数值模拟中的表现。最后将CBPS-AM方法分别应用于存在缺失的HIV临床试验数据和中国新冠病毒感染疫情数据中，建立科学的综合评价以及针对...     

一种异方差模型的两阶段估计

在异方差线性回归模型中,当模型误差项的协方差阵未知时,对异方差模型进行估计目前还没有比较好的方法。基于此,提出一种异方差模型的两阶段估计—基于异方差一致协方差阵估计,该方法将异方差一致协方差阵估计HC5m和广义最小二乘估计法结合起来,综合使用全部样本的信息,并对异方差模型进行估计。通过大量的蒙特卡洛数值模拟和实证分析,结果表明该方法具有一定的可行性和有效性。     

Regularized covariance matrix estimation under the common principal components model

The common principal components (CPC) model provides a way to model the population covariance matrices of several groups by assuming a common eigenvector structure. When appropriate, this model can provide covariance matrix estimators of which the elements have smaller standard errors than when using either the pooled covariance matrix or the per group unbiased sample covariance matrix estimators. In this article, a regularized CPC estimator under the assumption of a common (or partially common) eigenvector structure in the populations is proposed. After estimation of the common eigenvectors using the Flury–Gautschi (or other) algorithm, the off-diagonal elements of the nearly diagonalized covariance matrices are shrunk towards zero and multiplied with the orthogonal common eigenvector matrix to obtain the regularized CPC covariance matrix estimates. The optimal shrinkage intensity per group can be estimated using cross-validation. The efficiency of these estimators compared to the pooled and unbiased estimators is investigated in a Monte Carlo simulation study, and the regularized CPC estimator is applied to a real dataset to demonstrate the utility of the method.     

Double shrinkage estimators for large sparse covariance matrices

Covariance matrices play an important role in many multivariate techniques and hence a good covariance estimation is crucial in this kind of analysis. In many applications a sparse covariance matrix is expected due to the nature of the data or for simple interpretation. Hard thresholding, soft thresholding, and generalized thresholding were therefore developed to this end. However, these estimators do not always yield well-conditioned covariance estimates. To have sparse and well-conditioned estimates, we propose doubly shrinkage estimators: shrinking small covariances towards zero and then shrinking covariance matrix towards a diagonal matrix. Additionally, a richness index is defined to evaluate how rich a covariance matrix is. According to our simulations, the richness index serves as a good indicator to choose relevant covariance estimator.     

ON ESTIMATION OF THE POPULATION SPECTRAL DISTRIBUTION FROM A HIGH‐DIMENSIONAL SAMPLE COVARIANCE MATRIX

Sample covariance matrices play a central role in numerous popular statistical methodologies, for example principal components analysis, Kalman filtering and independent component analysis. However, modern random matrix theory indicates that, when the dimension of a random vector is not negligible with respect to the sample size, the sample covariance matrix demonstrates significant deviations from the underlying population covariance matrix. There is an urgent need to develop new estimation tools in such cases with high‐dimensional data to recover the characteristics of the population covariance matrix from the observed sample covariance matrix. We propose a novel solution to this problem based on the method of moments. When the parametric dimension of the population spectrum is finite and known, we prove that the proposed estimator is strongly consistent and asymptotically Gaussian. Otherwise, we combine the first estimation method with a cross‐validation procedure to select the unknown model dimension. Simulation experiments demonstrate the consistency of the proposed procedure. We also indicate possible extensions of the proposed estimator to the case where the population spectrum has a density.     

HIGH DIMENSIONAL COVARIANCE MATRIX ESTIMATION IN APPROXIMATE FACTOR MODELS

The variance covariance matrix plays a central role in the inferential theories of high dimensional factor models in finance and economics. Popular regularization methods of directly exploiting sparsity are not directly applicable to many financial problems. Classical methods of estimating the covariance matrices are based on the strict factor models, assuming independent idiosyncratic components. This assumption, however, is restrictive in practical applications. By assuming sparse error covariance matrix, we allow the presence of the cross-sectional correlation even after taking out common factors, and it enables us to combine the merits of both methods. We estimate the sparse covariance using the adaptive thresholding technique as in Cai and Liu (2011), taking into account the fact that direct observations of the idiosyncratic components are unavailable. The impact of high dimensionality on the covariance matrix estimation based on the factor structure is then studied.     

Factored principal components analysis, with applications to face recognition

A dimension reduction technique is proposed for matrix data, with applications to face recognition from images. In particular, we propose a factored covariance model for the data under study, estimate the parameters using maximum likelihood, and then carry out eigendecompositions of the estimated covariance matrix. We call the resulting method factored principal components analysis. We also develop a method for classification using a likelihood ratio criterion, which has previously been used for evaluating the strength of forensic evidence. The methodology is illustrated with applications in face recognition.     

Robust estimation of a high-dimensional integrated covariance matrix

In this article, we consider a robust method of estimating a realized covariance matrix calculated as the sum of cross products of intraday high-frequency returns. According to recent articles in financial econometrics, the realized covariance matrix is essentially contaminated with market microstructure noise. Although techniques for removing noise from the matrix have been studied since the early 2000s, they have primarily investigated a low-dimensional covariance matrix with statistically significant sample sizes. We focus on noise-robust covariance estimation under converse circumstances, that is, a high-dimensional covariance matrix possibly with a small sample size. For the estimation, we utilize a statistical hypothesis test based on the characteristic that the largest eigenvalue of the covariance matrix asymptotically follows a Tracy–Widom distribution. The null hypothesis assumes that log returns are not pure noises. If a sample eigenvalue is larger than the relevant critical value, then we fail to reject the null hypothesis. The simulation results show that the estimator studied here performs better than others as measured by mean squared error. The empirical analysis shows that our proposed estimator can be adopted to forecast future covariance matrices using real data.     

Recent developments in high dimensional covariance estimation and its related issues,a review

In this paper we review some of recent developments in high dimensional data analysis, especially in the estimation of covariance and precision matrix, asymptotic results on the eigenstructure in the principal components analysis, and some relevant issues such as test on the equality of two covariance matrices, determination of the number of principal components, and detection of hubs in a complex network.     

Correlation analysis-based error compensation recursive least-square identification method for the Hammerstein model

In this paper, the correlation analysis based error compensation recursive least-square (RLS) identification method is proposed for the Hammerstein model. Firstly, the covariance matrix between input and output data points of the Hammerstein model is derived by using separable signal to realize that the unmeasurable internal variable is replaced by the covariance matrix of input. Thus, the correlation analysis method can be accordingly used to estimate parameters of the linear part, which results in the identification problem of the nonlinear part separated from the linear part. In addition, a correction term is added to least-square estimation to compensate error caused by output noise, further the error compensation-based RLS method is obtained for the observed data from the Hammerstein model. As a result, the least-square identification method, which generates error in the presence of noise distribution, can be compensated. Finally, simulation experiments are conducted to illustrate the performance of the proposed identification method.     

Local linear regression with adaptive orthogonal fitting for the wind power application

Short-term forecasting of wind generation requires a model of the function for the conversion of meteorological variables (mainly wind speed) to power production. Such a power curve is nonlinear and bounded, in addition to being nonstationary. Local linear regression is an appealing nonparametric approach for power curve estimation, for which the model coefficients can be tracked with recursive Least Squares (LS) methods. This may lead to an inaccurate estimate of the true power curve, owing to the assumption that a noise component is present on the response variable axis only. Therefore, this assumption is relaxed here, by describing a local linear regression with orthogonal fit. Local linear coefficients are defined as those which minimize a weighted Total Least Squares (TLS) criterion. An adaptive estimation method is introduced in order to accommodate nonstationarity. This has the additional benefit of lowering the computational costs of updating local coefficients every time new observations become available. The estimation method is based on tracking the left-most eigenvector of the augmented covariance matrix. A robustification of the estimation method is also proposed. Simulations on semi-artificial datasets (for which the true power curve is available) underline the properties of the proposed regression and related estimation methods. An important result is the significantly higher ability of local polynomial regression with orthogonal fit to accurately approximate the target regression, even though it may hardly be visible when calculating error criteria against corrupted data.