Lp convergence and complete convergence for weighted sums of AANA random variables

Let {X_n, n ? 1} be a sequence of asymptotically almost negatively associated (AANA, for short) random variables which is stochastically dominated by a random variable X, and {d_ni, 1 ? i ? n, n ? 1} be a sequence of real function, which is defined on a compact set E. Under some suitable conditions, we investigate some convergence properties for weighted sums of AANA random variables, especially the L^p convergence and the complete convergence. As an application, the Marcinkiewicz–Zygmund-type strong law of large numbers for AANA random variables is obtained.     

Density Estimation by Total Variation Penalized Likelihood Driven by the Sparsity ℓ1 Information Criterion

Abstract. We propose a non‐linear density estimator, which is locally adaptive, like wavelet estimators, and positive everywhere, without a log‐ or root‐transform. This estimator is based on maximizing a non‐parametric log‐likelihood function regularized by a total variation penalty. The smoothness is driven by a single penalty parameter, and to avoid cross‐validation, we derive an information criterion based on the idea of universal penalty. The penalized log‐likelihood maximization is reformulated as an ?₁‐penalized strictly convex programme whose unique solution is the density estimate. A Newton‐type method cannot be applied to calculate the estimate because the ?₁‐penalty is non‐differentiable. Instead, we use a dual block coordinate relaxation method that exploits the problem structure. By comparing with kernel, spline and taut string estimators on a Monte Carlo simulation, and by investigating the sensitivity to ties on two real data sets, we observe that the new estimator achieves good L ₁ and L ₂ risk for densities with sharp features, and behaves well with ties.     

Asymptotics of Selective Inference

In this paper, we seek to establish asymptotic results for selective inference procedures removing the assumption of Gaussianity. The class of selection procedures we consider are determined by affine inequalities, which we refer to as affine selection procedures. Examples of affine selection procedures include selective inference along the solution path of the least absolute shrinkage and selection operator (LASSO), as well as selective inference after fitting the least absolute shrinkage and selection operator at a fixed value of the regularization parameter. We also consider some tests in penalized generalized linear models. Our result proves asymptotic convergence in the high‐dimensional setting where n<p, and n can be of a logarithmic factor of the dimension p for some procedures.     

Optimal estimation in functional linear regression for sparse noise‐contaminated data

In this article, we propose a novel approach to fit a functional linear regression in which both the response and the predictor are functions. We consider the case where the response and the predictor processes are both sparsely sampled at random time points and are contaminated with random errors. In addition, the random times are allowed to be different for the measurements of the predictor and the response functions. The aforementioned situation often occurs in longitudinal data settings. To estimate the covariance and the cross‐covariance functions, we use a regularization method over a reproducing kernel Hilbert space. The estimate of the cross‐covariance function is used to obtain estimates of the regression coefficient function and of the functional singular components. We derive the convergence rates of the proposed cross‐covariance, the regression coefficient, and the singular component function estimators. Furthermore, we show that, under some regularity conditions, the estimator of the coefficient function has a minimax optimal rate. We conduct a simulation study and demonstrate merits of the proposed method by comparing it to some other existing methods in the literature. We illustrate the method by an example of an application to a real‐world air quality dataset. The Canadian Journal of Statistics 47: 524–559; 2019 © 2019 Statistical Society of Canada     

On the exponential inequalities for widely orthant-dependent random variables

ABSTRACT

In this work, we establish some exponential inequalities for widely orthant-dependent random variables. We also obtain the convergence rate O(n^{? 1/2}ln?^1/2n) for the strong law of large numbers for widely orthant-dependent random variables.     

Wavelet-based estimation of regression function with strong mixing errors under fixed design

We consider wavelet-based non linear estimators, which are constructed by using the thresholding of the empirical wavelet coefficients, for the mean regression functions with strong mixing errors and investigate their asymptotic rates of convergence. We show that these estimators achieve nearly optimal convergence rates within a logarithmic term over a large range of Besov function classes B^s_{p, q}. The theory is illustrated with some numerical examples.

A new ingredient in our development is a Bernstein-type exponential inequality, for a sequence of random variables with certain mixing structure and are not necessarily bounded or sub-Gaussian. This moderate deviation inequality may be of independent interest.     

Joint covariate selection and joint subspace selection for multiple classification problems

We address the problem of recovering a common set of covariates that are relevant simultaneously to several classification problems. By penalizing the sum of ℓ ₂ norms of the blocks of coefficients associated with each covariate across different classification problems, similar sparsity patterns in all models are encouraged. To take computational advantage of the sparsity of solutions at high regularization levels, we propose a blockwise path-following scheme that approximately traces the regularization path. As the regularization coefficient decreases, the algorithm maintains and updates concurrently a growing set of covariates that are simultaneously active for all problems. We also show how to use random projections to extend this approach to the problem of joint subspace selection, where multiple predictors are found in a common low-dimensional subspace. We present theoretical results showing that this random projection approach converges to the solution yielded by trace-norm regularization. Finally, we present a variety of experimental results exploring joint covariate selection and joint subspace selection, comparing the path-following approach to competing algorithms in terms of prediction accuracy and running time.     

Geometric ergodicity of nonlinear autoregressive models with changing conditional variances

The authors give easy‐to‐check sufficient conditions for the geometric ergodicity and the finiteness of the moments of a random process x_t = ?(x_t‐1,…, x_t‐p) + ?_tσ(x_t‐1,…, x_t‐q) in which ?: R^p → R, σ R^q → R and (?_t) is a sequence of independent and identically distributed random variables. They deduce strong mixing properties for this class of nonlinear autoregressive models with changing conditional variances which includes, among others, the ARCH(p), the AR(p)‐ARCH(p), and the double‐threshold autoregressive models.     

Convergence properties for weighted sums of NSD random variables

Abstract

Let {X_n, n ? 1} be a sequence of negatively superadditive dependent (NSD, in short) random variables and {b_ni, 1 ? i ? n, n ? 1} be an array of real numbers. In this article, we study the strong law of large numbers for the weighted sums ∑ⁿ_{i = 1}b_niX_i without identical distribution. We present some sufficient conditions to prove the strong law of large numbers. As an application, the Marcinkiewicz-Zygmund strong law of large numbers for NSD random variables is obtained. In addition, the complete convergence for the weighted sums of NSD random variables is established. Our results generalize and improve some corresponding ones for independent random variables and negatively associated random variables.     

Bayesian Lasso-mixed quantile regression

In this paper, we discuss the regularization in linear-mixed quantile regression. A hierarchical Bayesian model is used to shrink the fixed and random effects towards the common population values by introducing an l₁ penalty in the mixed quantile regression check function. A Gibbs sampler is developed to simulate the parameters from the posterior distributions. Through simulation studies and analysis of an age-related macular degeneration (ARMD) data, we assess the performance of the proposed method. The simulation studies and the ARMD data analysis indicate that the proposed method performs well in comparison with the other approaches.     

Deviation of order p for estimators of the variance in first-order stochastic differential equation (SDE)

In this work, we consider a non-parametric estimator of the variance in one-dimensional diffusion models or, more generally, in Itô processes with a deterministic diffusion term and a general non-anticipative drift. The estimation is based on the quadratic variation of discrete time observations over a finite interval. In particular, a central limit theorem (CLT) is proved for the deviation in L ^p norm (p≥; 1) between the variance and this estimator. The method of the proof consists in writing the L ^p norm of the deviation, when the drift term is equal to zero, as a sum of 4-dependent random variables. The moments are then computed by means of a Gaussian approximation and a CLT for m-dependent random variables is applied. The convergence is stable in law, this allows the result for processes with general drifts to be obtained, by using Girsanov's formula.     

Non‐parametric Copula Estimation Under Bivariate Censoring

In this paper, we consider non‐parametric copula inference under bivariate censoring. Based on an estimator of the joint cumulative distribution function, we define a discrete and two smooth estimators of the copula. The construction that we propose is valid for a large range of estimators of the distribution function and therefore for a large range of bivariate censoring frameworks. Under some conditions on the tails of the distributions, the weak convergence of the corresponding copula processes is obtained in l^∞([0,1]²). We derive the uniform convergence rates of the copula density estimators deduced from our smooth copula estimators. Investigation of the practical behaviour of these estimators is performed through a simulation study and two real data applications, corresponding to different censoring settings. We use our non‐parametric estimators to define a goodness‐of‐fit procedure for parametric copula models. A new bootstrap scheme is proposed to compute the critical values.     

Laplace Error Penalty‐based Variable Selection in High Dimension

We propose the Laplace Error Penalty (LEP) function for variable selection in high‐dimensional regression. Unlike penalty functions using piecewise splines construction, the LEP is constructed as an exponential function with two tuning parameters and is infinitely differentiable everywhere except at the origin. With this construction, the LEP‐based procedure acquires extra flexibility in variable selection, admits a unified derivative formula in optimization and is able to approximate the L₀ penalty as close as possible. We show that the LEP procedure can identify relevant predictors in exponentially high‐dimensional regression with normal errors. We also establish the oracle property for the LEP estimator. Although not being convex, the LEP yields a convex penalized least squares function under mild conditions if p is no greater than n. A coordinate descent majorization‐minimization algorithm is introduced to implement the LEP procedure. In simulations and a real data analysis, the LEP methodology performs favorably among competitive procedures.     

Estimating conditional occupation-time distributions for dependent sequences

Consider a random integer-valued process X(t) on Z+ that satisfies some weak dependence condition. We study the empirical distribution function of the occupation times of such a process and prove convergence to a suitable Gaussian process. An application to the statistical analysis of open and closed sojourn-time distributions for ion channels is provided.     

Robust sparse regression and tuning parameter selection via the efficient bootstrap information criteria

There is currently much discussion about lasso-type regularized regression which is a useful tool for simultaneous estimation and variable selection. Although the lasso-type regularization has several advantages in regression modelling, owing to its sparsity, it suffers from outliers because of using penalized least-squares methods. To overcome this issue, we propose a robust lasso-type estimation procedure that uses the robust criteria as the loss function, imposing L₁-type penalty called the elastic net. We also introduce to use the efficient bootstrap information criteria for choosing optimal regularization parameters and a constant in outlier detection. Simulation studies and real data analysis are given to examine the efficiency of the proposed robust sparse regression modelling. We observe that our modelling strategy performs well in the presence of outliers.     

An Asymptotic Linear Representation for the Breslow Estimator

We provide an asymptotic linear representation for the Breslow estimator of the baseline cumulative hazard function in the Cox model. Our representation consists of an average of independent random variables and a term involving the difference between the maximum partial likelihood estimator and the underlying regression parameter. The order of the remainder term is arbitrarily close to n ^?1.     

Model-averaged ℓ1 regularization using Markov chain Monte Carlo model composition

Bayesian model averaging (BMA) is an effective technique for addressing model uncertainty in variable selection problems. However, current BMA approaches have computational difficulty dealing with data in which there are many more measurements (variables) than samples. This paper presents a method for combining ?₁ regularization and Markov chain Monte Carlo model composition techniques for BMA. By treating the ?₁ regularization path as a model space, we propose a method to resolve the model uncertainty issues arising in model averaging from solution path point selection. We show that this method is computationally and empirically effective for regression and classification in high-dimensional data sets. We apply our technique in simulations, as well as to some applications that arise in genomics.     

Testing for a Change of the Innovation Distribution in Nonparametric Autoregression: The Sequential Empirical Process Approach

We consider a nonparametric autoregression model under conditional heteroscedasticity with the aim to test whether the innovation distribution changes in time. To this end, we develop an asymptotic expansion for the sequential empirical process of nonparametrically estimated innovations (residuals). We suggest a Kolmogorov–Smirnov statistic based on the difference of the estimated innovation distributions built from the first ?ns?and the last n ? ?ns? residuals, respectively (0 ≤ s ≤ 1). Weak convergence of the underlying stochastic process to a Gaussian process is proved under the null hypothesis of no change point. The result implies that the test is asymptotically distribution‐free. Consistency against fixed alternatives is shown. The small sample performance of the proposed test is investigated in a simulation study and the test is applied to a data example.     

A general non-central hypergeometric distribution

We construct a general non-central hypergeometric distribution, which models biased sampling without replacement. Our distribution is constructed from the combined order statistics of two samples: one of independent and identically distributed random variables with absolutely continuous distribution F and the other of independent and identically distributed random variables with absolutely continuous distribution G. The distribution depends on F and G only through F○G^{( ? 1)} (F composed with the quantile function of G), and the standard hypergeometric distribution and Wallenius’ non-central hypergeometric distribution arise as special cases. We show in efficient economic markets the quantity traded has a general non-central hypergeometric distribution.     

Change-point detection in a linear model by adaptive fused quantile method

A novel approach to quantile estimation in multivariate linear regression models with change-points is proposed: the change-point detection and the model estimation are both performed automatically, by adopting either the quantile-fused penalty or the adaptive version of the quantile-fused penalty. These two methods combine the idea of the check function used for the quantile estimation and the L₁ penalization principle known from the signal processing and, unlike some standard approaches, the presented methods go beyond typical assumptions usually required for the model errors, such as sub-Gaussian or normal distribution. They can effectively handle heavy-tailed random error distributions, and, in general, they offer a more complex view on the data as one can obtain any conditional quantile of the target distribution, not just the conditional mean. The consistency of detection is proved and proper convergence rates for the parameter estimates are derived. The empirical performance is investigated via an extensive comparative simulation study and practical utilization is demonstrated using a real data example.