Least-square regularized regression with non-iid sampling

We study the least-square regression learning algorithm generated by regularization schemes in reproducing kernel Hilbert spaces. A non-iid setting is considered: the sequence of probability measures for sampling is not identical and the sampling may be dependent. When the sequence of marginal distributions for sampling converges exponentially fast in the dual of a Hölder space and the sampling process satisfies a polynomial strong mixing condition, we derive learning rates for the learning algorithm.     

Threshold Estimation for Stochastic Processes with Small Noise

Consider a process satisfying a stochastic differential equation with unknown drift parameter, and suppose that discrete observations are given. It is known that a simple least squares estimator (LSE) can be consistent but numerically unstable in the sense of large standard deviations under finite samples when the noise process has jumps. We propose a filter to cut large shocks from data and construct the same LSE from data selected by the filter. The proposed estimator can be asymptotically equivalent to the usual LSE, whose asymptotic distribution strongly depends on the noise process. However, in numerical study, it looked asymptotically normal in an example where filter was chosen suitably, and the noise was a Lévy process. We will try to justify this phenomenon mathematically, under certain restricted assumptions.     

Improving variance function estimation in semiparametric longitudinal data analysis

We propose an efficient and robust method for variance function estimation in semiparametric longitudinal data analysis. The method utilizes a local log‐linear approximation for the variance function and adopts a generalized estimating equation approach to account for within subject correlations. We show theoretically and empirically that our method outperforms estimators using working independence that ignores the correlations. The Canadian Journal of Statistics 39: 656–670; 2011. © 2011 Statistical Society of Canada     

Noninformative priors for linear combinations of normal means with unequal variances

For normal populations with unequal variances, we develop matching priors and reference priors for a linear combination of the means. Here, we find three second-order matching priors: a highest posterior density (HPD) matching prior, a cumulative distribution function (CDF) matching prior, and a likelihood ratio (LR) matching prior. Furthermore, we show that the reference priors are all first-order matching priors, but that they do not satisfy the second-order matching criterion that establishes the symmetry and the unimodality of the posterior under the developed priors. The results of a simulation indicate that the second-order matching prior outperforms the reference priors in terms of matching the target coverage probabilities, in a frequentist sense. Finally, we compare the Bayesian credible intervals based on the developed priors with the confidence intervals derived from real data.     

A Comparison of Survival Function Estimators in the Koziol–Green Model

In this paper, three competing survival function estimators are compared under the assumptions of the so-called Koziol– Green model, which is a simple model of informative random censoring. It is shown that the model specific estimators of Ebrahimi and Abdushukurov, Cheng, and Lin are asymptotically equivalent. Further, exact expressions for the (noncentral) moments of these estimators are given, and their biases are analytically compared with the bias of the familiar Kaplan–Meier estimator. Finally, MSE comparisons of the three estimators are given for some selected rates of censoring.     

Modified minimax estimation of regression coefficients

Working with the linear regression model (1.1) and having the extraneous information (1.2) about regression coefficients the problem exists how to build estimators (1.3) with the risk (1.4) which enable to utilize the known information in order to reduce their risk as compared with the risk (1.6) of the LSE (1.5). Solution of this problem is known for the positive definite matrix T, namely in form for estimators (1.8) and (1.10).First, it is shown that the proposed estimators (2.6),(2.9) and (2.16) based on psedoinversions of the matrix L represent the solution of the problem of the positive semidefinite matrix T=L'L.Further, the problem of interpretability of estimators in the sense of the inequality (3.1) exists; it is shown that all mentioned estimators are at least partially interpretable in the sense of requirements (3.2) or (3.10).     

Methods for calculating stationary distribution in linear models of time series

The article deals with methods for computing the stationary marginal distribution in linear models of time series. Two approaches are described. First, an algorithm based on approximation of solution of the corresponding integral equation is briefly reviewed. Then, we study the limit behaviour of the partial sums c ₁ η₁+c ₂ η₂+···+c _n η _n where η _i are i.i.d. random variables and c _i real constants. We generalize procedure of Haiman (1998) [Haiman, G., 1998, Upper and lower bounds for the tail of the invariant distribution of some AR(1) processes. Asymptotic Methods in Probability and Statistics, 45, 723–730.] to an arbitrary causal linear process and relax the assumptions of his result significantly. This is achieved by investigating the properties of convolution of densities.     

Nonparametric regression with cross‐classified responses

In this article, we develop regression models with cross‐classified responses. Conditional independence structures can be explored/exploited through the selective inclusion/exclusion of terms in a certain functional ANOVA decomposition, and the estimation is done nonparametrically via the penalized likelihood method. A cohort of computational and data analytical tools are presented, which include cross‐validation for smoothing parameter selection, Kullback–Leibler projection for model selection, and Bayesian confidence intervals for odds ratios. Random effects are introduced to model possible correlations such as those found in longitudinal and clustered data. Empirical performances of the methods are explored in simulation studies of limited scales, and a real data example is presented using some eyetracking data from linguistic studies. The techniques are implemented in a suite of R functions, whose usage is briefly described in the appendix. The Canadian Journal of Statistics 39: 591–609; 2011. © 2011 Statistical Society of Canada