Averaging estimation for conditional covariance models

Estimating conditional covariance matrices is important in statistics and finance. In this paper, we propose an averaging estimator for the conditional covariance, which combines the estimates of marginal conditional covariance matrices by Model Averaging MArginal Regression of Li, Linton, and Lu. This estimator avoids the “curse of dimensionality” problem that the local constant estimator of Yin et al. suffered from. We establish the asymptotic properties of the averaging weights and that of the proposed conditional covariance estimator. The finite sample performances are augmented by simulation. An application to portfolio allocation illustrates the practical superiority of the averaging estimator.     

Semiparametric statistical inferences for longitudinal data with nonparametric covariance modelling

In this paper, semiparametric modelling for longitudinal data with an unstructured error process is considered. We propose a partially linear additive regression model for longitudinal data in which within-subject variances and covariances of the error process are described by unknown univariate and bivariate functions, respectively. We provide an estimating approach in which polynomial splines are used to approximate the additive nonparametric components and the within-subject variance and covariance functions are estimated nonparametrically. Both the asymptotic normality of the resulting parametric component estimators and optimal convergence rate of the resulting nonparametric component estimators are established. In addition, we develop a variable selection procedure to identify significant parametric and nonparametric components simultaneously. We show that the proposed SCAD penalty-based estimators of non-zero components have an oracle property. Some simulation studies are conducted to examine the finite-sample performance of the proposed estimation and variable selection procedures. A real data set is also analysed to demonstrate the usefulness of the proposed method.     

Variance estimation in nonparametric regression with jump discontinuities

Variance estimation is an important topic in nonparametric regression. In this paper, we propose a pairwise regression method for estimating the residual variance. Specifically, we regress the squared difference between observations on the squared distance between design points, and then estimate the residual variance as the intercept. Unlike most existing difference-based estimators that require a smooth regression function, our method applies to regression models with jump discontinuities. Our method also applies to the situations where the design points are unequally spaced. Finally, we conduct extensive simulation studies to evaluate the finite-sample performance of the proposed method and compare it with some existing competitors.     

Partial deconvolution estimation in nonparametric regression

In this article, we propose a class of partial deconvolution kernel estimators for the nonparametric regression function when some covariates are measured with error and some are not. The estimation procedure combines the classical kernel methodology and the deconvolution kernel technique. According to whether the measurement error is ordinarily smooth or supersmooth, we establish the optimal local and global convergence rates for these proposed estimators, and the optimal bandwidths are also identified. Furthermore, lower bounds for the convergence rates of all possible estimators for the nonparametric regression functions are developed. It is shown that, in both the super and ordinarily smooth cases, the convergence rates of the proposed partial deconvolution kernel estimators attain the lower bound. The Canadian Journal of Statistics 48: 535–560; 2020 © 2020 Statistical Society of Canada     

Rank covariance matrix estimation of a partially known covariance matrix

Classical multivariate methods are often based on the sample covariance matrix, which is very sensitive to outlying observations. One alternative to the covariance matrix is the affine equivariant rank covariance matrix (RCM) that has been studied in Visuri et al. [2003. Affine equivariant multivariate rank methods. J. Statist. Plann. Inference 114, 161–185]. In this article we assume that the covariance matrix is partially known and study how to estimate the corresponding RCM. We use the properties that the RCM is affine equivariant and that the RCM is proportional to the inverse of the regular covariance matrix, and hence reduce the problem of estimating the original RCM to estimating marginal rank covariance matrices. This is a great computational advantage when the dimension of the original data vector is large.     

Functional data: local linear estimation of the conditional density and its application

In this paper, we introduce a new nonparametric estimation procedure of the conditional density of a scalar response variable given a random variable taking values in a semi-metric space. Under some general conditions, we establish both the pointwise and the uniform almost-complete consistencies with convergence rates of the conditional density estimator related to this estimation procedure. Moreover, we give some particular cases of our results which can also be considered as novel in the finite-dimensional setting. Notice also that the results of this paper are used to derive some asymptotic properties of the local linear estimator of the conditional mode.     

A permutation-based Bayesian approach for inverse covariance estimation

Abstract

Covariance estimation and selection for multivariate datasets in a high-dimensional regime is a fundamental problem in modern statistics. Gaussian graphical models are a popular class of models used for this purpose. Current Bayesian methods for inverse covariance matrix estimation under Gaussian graphical models require the underlying graph and hence the ordering of variables to be known. However, in practice, such information on the true underlying model is often unavailable. We therefore propose a novel permutation-based Bayesian approach to tackle the unknown variable ordering issue. In particular, we utilize multiple maximum a posteriori estimates under the DAG-Wishart prior for each permutation, and subsequently construct the final estimate of the inverse covariance matrix. The proposed estimator has smaller variability and yields order-invariant property. We establish posterior convergence rates under mild assumptions and illustrate that our method outperforms existing approaches in estimating the inverse covariance matrices via simulation studies.     

Uniform strong convergence results for the conditional kaplan-meier estimator and its quantiles

We consider a fixed design model in which the responses are possibly right censored. The aim of this paper is to establish some important almost sure convergence properties of the Kaplan-Meier type estimator for the lifetime distribution at a given covariate value. We also consider the corresponding quantile estimator and obtain a modulus of continuity result. Our rates of uniform strong convergence are obtained via exponential probability bounds.     

Change points in nonparametric regresion functions

Estimators of location and size of jumps or discontinuities in a regression function and/or its derivatives are proposed. The estimators are based on the analysis of residuals obtained from the locally weighted least squares regression. The proposed estimators adapt to both fixed and random designs. The asymptotic properties of the estimators are investigated. The method is illustrated through simulation studies.     

Effects of the estimation of covariance matrix parameters in the generalized multivariate linear model

We Consider the generalized multivariate linear model and assume the covariance matrix of the p x 1 vector of responses on a given individual can be represented in the general linear structure form described by Anderson (1973). The effects of the use of estimates of the parameters of the covariance matrix on the generalized least squares estimator of the regression coefficients and on the prediction of a portion of a future vector, when only the first portion of the vector has been observed, are investigated. Approximations are derived for the covariance matrix of the generalized least squares estimator and for the mean square error matrix of the usual predictor, for the practical case where estimated parameters are used.     

A note on nonparametric estimation of circular conditional densities

ABSTRACT

The conditional density offers the most informative summary of the relationship between explanatory and response variables. We need to estimate it in place of the simple conditional mean when its shape is not well-behaved. A motivation for estimating conditional densities, specific to the circular setting, lies in the fact that a natural alternative of it, like quantile regression, could be considered problematic because circular quantiles are not rotationally equivariant. We treat conditional density estimation as a local polynomial fitting problem as proposed by Fan et al. [Estimation of conditional densities and sensitivity measures in nonlinear dynamical systems. Biometrika. 1996;83:189–206] in the Euclidean setting, and discuss a class of estimators in the cases when the conditioning variable is either circular or linear. Asymptotic properties for some members of the proposed class are derived. The effectiveness of the methods for finite sample sizes is illustrated by simulation experiments and an example using real data.     

Multilevel maximum likelihood estimation with application to covariance matrices

The asymptotic variance of the maximum likelihood estimate is proved to decrease when the maximization is restricted to a subspace that contains the true parameter value. Maximum likelihood estimation allows a systematic fitting of covariance models to the sample, which is important in data assimilation. The hierarchical maximum likelihood approach is applied to the spectral diagonal covariance model with different parameterizations of eigenvalue decay, and to the sparse inverse covariance model with specified parameter values on different sets of nonzero entries. It is shown computationally that using smaller sets of parameters can decrease the sampling noise in high dimension substantially.     

Adaptive nonparametric estimation in the presence of dependence

We consider nonparametric estimation problems in the presence of dependent data, notably nonparametric regression with random design and nonparametric density estimation. The proposed estimation procedure is based on a dimension reduction. The minimax optimal rate of convergence of the estimator is derived assuming a sufficiently weak dependence characterised by fast decreasing mixing coefficients. We illustrate these results by considering classical smoothness assumptions. However, the proposed estimator requires an optimal choice of a dimension parameter depending on certain characteristics of the function of interest, which are not known in practice. The main issue addressed in our work is an adaptive choice of this dimension parameter combining model selection and Lepski's method. It is inspired by the recent work of Goldenshluger and Lepski [(2011), ‘Bandwidth Selection in Kernel Density Estimation: Oracle Inequalities and Adaptive Minimax Optimality’, The Annals of Statistics, 39, 1608–1632]. We show that this data-driven estimator can attain the lower risk bound up to a constant provided a fast decay of the mixing coefficients.     

The exact density of nonparametric regression estimators: fixed design case

This paper studies the exact density of a general nonparametric regression estimator when the errors are non-normal. The fixed design case is considered. The density function is derived by an application of the technique of Davis (1976)     

Testing discontinuities in nonparametric regression

In nonparametric regression, it is often needed to detect whether there are jump discontinuities in the mean function. In this paper, we revisit the difference-based method in [13] and propose to further improve it. To achieve the goal, we first reveal that their method is less efficient due to the inappropriate choice of the response variable in their linear regression model. We then propose a new regression model for estimating the residual variance and the total amount of discontinuities simultaneously. In both theory and simulation, we show that the proposed variance estimator has a smaller mean-squared error compared to the existing estimator, whereas the estimation efficiency for the total amount of discontinuities remains unchanged. Finally, we construct a new test procedure for detection of discontinuities using the proposed method; and via simulation studies, we demonstrate that our new test procedure outperforms the existing one in most settings.     

A consistent estimator for the supremum of the projection index based on sliced inverse regression

To seek the nonlinear structure hidden in data points of high-dimension, a transformation related to projection pursuit method and a projection index were proposed by Li (1989, 1990 ). In this paper, we present a consistent estimator of the supremum of the projection index based sliced inverse regression technique. This estimator also suggests a method to obtain approximately the most interesting projection in the general case.     

Plug-in bandwidth choice for estimation of nonparametric part in partial linear regression models with strong mixing errors

Consider a regression model where the regression function is the sum of a linear and a nonparametric component. Assuming that the errors of the model follow a stationary strong mixing process with mean zero, the problem of bandwidth selection for a kernel estimator of the nonparametric component is addressed here. We obtain an asymptotic expression for an optimal band-width and we propose to use a plug-in methodology in order to estimate this bandwidth through preliminary estimates of the unknown quantities. Asymptotic optimality for the plug-in bandwidth is established.     

Dimension reduction for the conditional kth moment in regression

The idea of dimension reduction without loss of information can be quite helpful for guiding the construction of summary plots in regression without requiring a prespecified model. Central subspaces are designed to capture all the information for the regression and to provide a population structure for dimension reduction. Here, we introduce the central k th-moment subspace to capture information from the mean, variance and so on up to the k th conditional moment of the regression. New methods are studied for estimating these subspaces. Connections with sliced inverse regression are established, and examples illustrating the theory are presented.