Empirical Bayes regression analysis with many regressors but fewer observations

In this paper, we consider the prediction problem in multiple linear regression model in which the number of predictor variables, p, is extremely large compared to the number of available observations, n  . The least-squares predictor based on a generalized inverse is not efficient. We propose six empirical Bayes estimators of the regression parameters. Three of them are shown to have uniformly lower prediction error than the least-squares predictors when the vector of regressor variables are assumed to be random with mean vector zero and the covariance matrix (1/n)X^tX$(1/n){\mathit{X}}^{\mathrm{t}}\mathit{X}$ where X^t=(_x1,…,_xn)${\mathit{X}}^{\mathrm{t}}=({\mathit{x}}_1,\dots ,{\mathit{x}}_n)$ is the p×n$p\times n$ matrix of observations on the regressor vector centered from their sample means. For other estimators, we use simulation to show its superiority over the least-squares predictor.     

Nonparametric estimation of conditional expectation

Denote the integer lattice points in the N  -dimensional Euclidean space by Z^N${\mathbb{Z}}^N$ and assume that (_Xi,_Yi)$(X_{\mathbf{i}},Y_{\mathbf{i}})$, i∈Z^N$\mathbf{i}\in {\mathbb{Z}}^N$ is a mixing random field. Estimators of the conditional expectation r(x)=E[_Yi|_Xi=x]$r(\mathbf{x})=\mathrm{E}[Y_{\mathbf{i}}|X_{\mathbf{i}}=\mathbf{x}]$ by nearest neighbor methods are established and investigated. The main analytical result of this study is that, under general mixing assumptions, the estimators considered are asymptotically normal. Many difficulties arise since points in higher dimensional space N?2$N?2$ cannot be linearly ordered. Our result applies to many situations where parametric methods cannot be adopted with confidence.     

Prediction in moving average processes

For the stationary invertible moving average process of order one with unknown innovation distribution F, we construct root-n   consistent plug-in estimators of conditional expectations E(h(_Xn+1)|_X1,…,_Xn)$E(h(X_{n+1})|X_1,\dots ,X_n)$. More specifically, we give weak conditions under which such estimators admit Bahadur-type representations, assuming some smoothness of h or of F. For fixed h it suffices that h   is locally of bounded variation and locally Lipschitz in _L2(F)$L_2(F)$, and that the convolution of h and F   is continuously differentiable. A uniform representation for the plug-in estimator of the conditional distribution function P(_Xn+1?·|_X1,…,_Xn)$P(X_{n+1}?·|X_1,\dots ,X_n)$ holds if F has a uniformly continuous density. For a smoothed version of our estimator, the Bahadur representation holds uniformly over each class of functions h that have an appropriate envelope and whose shifts are F-Donsker, assuming some smoothness of F. The proofs use empirical process arguments.     

Testing of a sub-hypothesis in linear regression models with long memory errors and deterministic design

This paper considers the problem of testing a sub-hypothesis in homoscedastic linear regression models where errors form long memory moving average processes and designs are non-random. Unlike in the random design case, asymptotic null distribution of the likelihood ratio type test based on the Whittle quadratic form is shown to be non-standard and non-chi-square. Moreover, the rate of consistency of the minimum Whittle dispersion estimator of the slope parameter vector is shown to be n^-(1-α)/2$n^{-(1-\alpha )/2}$, different from the rate n^-1/2$n^{-1/2}$ obtained in the random design case, where α$\alpha$ is the rate at which the error spectral density explodes at the origin. The proposed test is shown to be consistent against fixed alternatives and has non-trivial asymptotic power against local alternatives that converge to null hypothesis at the rate n^-(1-α)/2$n^{-(1-\alpha )/2}$.     

Regression models for functional data by reproducing kernel Hilbert spaces methods

Non-parametric regression models are developed when the predictor is a function-valued random variable X={_Xt}t∈T$X=\{X_t{\}}_{t\in T}$. Based on a representation of the regression function f(X)$f(X)$ in a reproducing kernel Hilbert space such models generalize the classical setting used in statistical learning theory. Two applications corresponding to scalar and categorical response random variable are performed on stock-exchange and medical data. The results of different regression models are compared.     

Characterizations based on record values and order statistics

Consider a sequence of independent and identically distributed random variables {_Xi,i?1}$\{X_i,i?1\}$ with a common absolutely continuous distribution function F  . Let _X1:n?_X2:n???_Xn:n$X_{1:n}?X_{2:n}???X_{n:n}$ be the order statistics of {_X1,_X2,…,_Xn}$\{X_1,X_2,\dots ,X_n\}$ and {_Yl,l?1}$\{Y_l,l?1\}$ be the sequence of record values generated by {_Xi,i?1}$\{X_i,i?1\}$. In this work, the conditional distribution of _Yl$Y_l$ given _Xn:n$X_{n:n}$ is established. Some characterizations of F   based on record values and _Xn:n$X_{n:n}$ are then given.     

Multivariate orthogonal series estimates for random design regression

In this paper a new multivariate regression estimate is introduced. It is based on ideas derived in the context of wavelet estimates and is constructed by hard thresholding of estimates of coefficients of a series expansion of the regression function. Multivariate functions constructed analogously to the classical Haar wavelets are used for the series expansion. These functions are orthogonal in _L2(_μn)$L_2({\mu }_n)$, where _μn${\mu }_n$ denotes the empirical design measure. The construction can be considered as designing adapted Haar wavelets.     

Combinatorially composing Chebyshev polynomials

We present a combinatorial proof of two fundamental composition identities associated with Chebyshev polynomials. Namely, for all m,n?0$m,n?0$, _Tm(_Tn(x))=_Tmn(x)$T_m(T_n(x))=T_{\mathit{mn}}(x)$ and _Um−1(_Tn(x))_Un−1(x)=_Umn−1(x).$U_{m-1}(T_n(x))U_{n-1}(x)=U_{\mathit{mn}-1}(x).$