On the Strong Consistency of Ridge Estimates

In this article, we establish strong consistency of the ridge estimates using extended results for the strong consistency of the least squares estimates in multiple regression models which discard the usual assumption of null mean value for the errors and only requires them to be i.i.d. with absolute moment of order r (0 < r ? 1).     

Strong consistency of least squares estimates with i.i.d. errors with mean values not necessarily defined

We establish strong consistency of the least squares estimates in multiple regression models discarding the usual assumption of the errors having null mean value. Thus, we required them to be i.i.d. with absolute moment of order r, 0<r<2, and null mean value when r>1. Only moderately restrictive conditions are imposed on the model matrix. In our treatment, we use an extension of the Marcinkiewicz–Zygmund strong law to overcome the errors mean value not being defined. In this way, we get a unified treatment for the case of i.i.d. errors extending the results of some previous papers.     

A Note on Nonparametric Regression with β-Mixing Sequences

Statistical risks of least-squares estimates of a regression function (possibly nonlinear) are studied under a β-mixing setup. The assumptions are minimal and are virtually the same as those of the i.i.d. case. The bounds obtained are optimal, up to a term of order log n. Similar results are also obtained for complexity regularized regression estimates in the presence of penalty functions.     

Non-parametric regression for spatially dependent data with wavelets

We study non-parametric regression estimates for random fields. The data satisfies certain strong mixing conditions and is defined on the regular N-dimensional lattice structure. We show consistency and obtain rates of convergence. The rates are optimal modulo a logarithmic factor in some cases. As an application, we estimate the regression function with multidimensional wavelets which are not necessarily isotropic. We simulate random fields on planar graphs with the concept of concliques (cf. [Kaiser MS, Lahiri SN, Nordman DJ. Goodness of fit tests for a class of markov random field models. Ann Statist. 2012;40:104–130]) in numerical examples of the estimation procedure.     

Model selection consistency of U-statistics with convex loss and weighted lasso penalty

In the paper we consider minimisation of U-statistics with the weighted Lasso penalty and investigate their asymptotic properties in model selection and estimation. We prove that the use of appropriate weights in the penalty leads to the procedure that behaves like the oracle that knows the true model in advance, i.e. it is model selection consistent and estimates nonzero parameters with the standard rate. For the unweighted Lasso penalty, we obtain sufficient and necessary conditions for model selection consistency of estimators. The obtained results strongly based on the convexity of the loss function that is the main assumption of the paper. Our theorems can be applied to the ranking problem as well as generalised regression models. Thus, using U-statistics we can study more complex models (better describing real problems) than usually investigated linear or generalised linear models.     

On the predictive performance of the almost unbiased Liu estimator

ABSTRACT

Regression models are usually used in forecasting (predicting) unknown values of the response variable y. This article considers the predictive performance of the almost unbiased Liu estimator compared to the ordinary least-squares estimator, principal component regression estimator, and Liu estimator. Finally, we present a numerical example to explain the theoretical results and we obtain a region where the almost unbiased Liu estimator is uniformly superior to the ordinary least-squares estimator, principal component regression estimator, and Liu estimator.     

Asymptotics of M-estimator in multivariate linear regression models for a class of random errors

It is known that linear regression models have immense applications in various areas such as engineering technology, economics and social sciences. In this paper, we investigate the asymptotic properties of M-estimator in multivariate linear regression model based on a class of random errors satisfying a generalised Bernstein-type inequality. By using the generalised Bernstein-type inequality, we obtain a general result on almost sure convergence for a class of random variables and then obtain the strong consistency for the M-estimator in multivariate linear regression models under some mild conditions. The result extends or improves some existing ones in the literature. Moreover, we also consider the case when the dimension $p$ tends to infinity by establishing the rate of almost sure convergence for a class of random variables satisfying generalised Bernstein-type inequality. Some numerical simulations are also provided to verify the validity of the theoretical results.     

Asymptotic properties for LS estimators in EV regression model with dependent errors

In this paper, we establish the strong consistency and asymptotic normality for the least square (LS) estimators in simple linear errors-in-variables (EV) regression models when the errors form a stationary α-mixing sequence of random variables. The quadratic-mean consistency is also considered.     

Skewness by Splitting the Scale Parameter

This article describes a recursive nonparametric estimation for the local partial first derivative of an arbitrary function satisfied some regularity conditions and establishes its consistency and asymptotic normality under the assumption of strong mixing sequence. The proposed estimator is a variable window width version of the Watson-Nadaraya type of derivative estimator. The window width varied as more data points become available enables a recursive algorithm that reduce computational complexity from order N ³ normally required by batch methods for kernel regression to order N ². This approach is computationally simple and attractive from practical viewpoint especially when the situation call for frequent updating of first derivative estimates. For example, maintaining a delta-hedged position of a portfolio of equities with index options is one of many applications of such estimation.     

Robust quasi-likelihood inference in generalized linear mixed models with outliers

It is well known that in a traditional outlier-free situation, the generalized quasi-likelihood (GQL) approach [B.C. Sutradhar, On exact quasilikelihood inference in generalized linear mixed models, Sankhya: Indian J. Statist. 66 (2004), pp. 261–289] performs very well to obtain the consistent as well as the efficient estimates for the parameters involved in the generalized linear mixed models (GLMMs). In this paper, we first examine the effect of the presence of one or more outliers on the GQL estimation for the parameters in such GLMMs, especially in two important models such as count and binary mixed models. The outliers appear to cause serious biases and hence inconsistency in the estimation. As a remedy, we then propose a robust GQL (RGQL) approach in order to obtain the consistent estimates for the parameters in the GLMMs in the presence of one or more outliers. An extensive simulation study is conducted to examine the consistency performance of the proposed RGQL approach.     

Moment Consistency of the Exchangeably Weighted Bootstrap for Semiparametric M‐estimation

The bootstrap variance estimate is widely used in semiparametric inferences. However, its theoretical validity is a well‐known open problem. In this paper, we provide a first theoretical study on the bootstrap moment estimates in semiparametric models. Specifically, we establish the bootstrap moment consistency of the Euclidean parameter, which immediately implies the consistency of t‐type bootstrap confidence set. It is worth pointing out that the only additional cost to achieve the bootstrap moment consistency in contrast with the distribution consistency is to simply strengthen the L₁ maximal inequality condition required in the latter to the L_p maximal inequality condition for p≥1. The general L_p multiplier inequality developed in this paper is also of independent interest. These general conclusions hold for the bootstrap methods with exchangeable bootstrap weights, for example, non‐parametric bootstrap and Bayesian bootstrap. Our general theory is illustrated in the celebrated Cox regression model.     

Adaptation of the jackknifed ridge methods to the linear mixed models

The purpose of this article is to obtain the jackknifed ridge predictors in the linear mixed models and to examine the superiorities, the linear combinations of the jackknifed ridge predictors over the ridge, principal components regression, r?k class and Henderson's predictors in terms of bias, covariance matrix and mean square error criteria. Numerical analyses are considered to illustrate the findings and a simulation study is conducted to see the performance of the jackknifed ridge predictors.     

Estimation of patterned covariance in the multivariate linear models: an outer product least-squares approach

In this article, we present a framework of estimating patterned covariance of interest in the multivariate linear models. The main idea in it is to estimate a patterned covariance by minimizing a trace distance function between outer product of residuals and its expected value. The proposed framework can provide us explicit estimators, called outer product least-squares estimators, for parameters in the patterned covariance of the multivariate linear model without or with restrictions on regression coefficients. The outer product least-squares estimators enjoy the desired properties in finite and large samples, including unbiasedness, invariance, consistency and asymptotic normality. We still apply the framework to three special situations where their patterned covariances are the uniform correlation, a generalized uniform correlation and a general q-dependence structure, respectively. Simulation studies for three special cases illustrate that the proposed method is a competent alternative of the maximum likelihood method in finite size samples.     

Local influence in ridge semiparametric models

ABSTRACT

Ridge penalized least-squares estimators has been suggested as an alternative to the minimum penalized sum of squares estimates in the presence of collinearity among the explanatory variables in semiparametric regression models (SPRMs). This paper studies the local influence of minor perturbations on the ridge estimates in the SPRM. The diagnostics under the perturbation of ridge penalized sum of squares, response variable, explanatory variables and ridge parameter are considered. Some local influence diagnostics are given. A Monte Carlo simulation study and a real example are used to illustrate the proposed perturbations.     

Change-point detection in a linear model by adaptive fused quantile method

A novel approach to quantile estimation in multivariate linear regression models with change-points is proposed: the change-point detection and the model estimation are both performed automatically, by adopting either the quantile-fused penalty or the adaptive version of the quantile-fused penalty. These two methods combine the idea of the check function used for the quantile estimation and the L₁ penalization principle known from the signal processing and, unlike some standard approaches, the presented methods go beyond typical assumptions usually required for the model errors, such as sub-Gaussian or normal distribution. They can effectively handle heavy-tailed random error distributions, and, in general, they offer a more complex view on the data as one can obtain any conditional quantile of the target distribution, not just the conditional mean. The consistency of detection is proved and proper convergence rates for the parameter estimates are derived. The empirical performance is investigated via an extensive comparative simulation study and practical utilization is demonstrated using a real data example.     

Bias correction for exponential family nonlinear modles

In this paper we discuss the computation of the bias to order n ^-1 for the parameter estimates in a general class of nonlinear regression models. Simple formulae are given to some special models. Diagnostic methods to assess the relationship between bias and observations are presented. Finally the proposed methods are illustrated by two examples.     

Improved Score Tests in Symmetric Linear Regression Models

The class of symmetric linear regression models has the normal linear regression model as a special case and includes several models that assume that the errors follow a symmetric distribution with longer-than-normal tails. An important member of this class is the t linear regression model, which is commonly used as an alternative to the usual normal regression model when the data contain extreme or outlying observations. In this article, we develop second-order asymptotic theory for score tests in this class of models. We obtain Bartlett-corrected score statistics for testing hypotheses on the regression and the dispersion parameters. The corrected statistics have chi-squared distributions with errors of order O(n ^?3/2), n being the sample size. The corrections represent an improvement over the corresponding original Rao's score statistics, which are chi-squared distributed up to errors of order O(n ^?1). Simulation results show that the corrected score tests perform much better than their uncorrected counterparts in samples of small or moderate size.     

On the asymptotic distribution of an estimate of the change point in a failure rate

The failure rate r(t) is assumed to have the shape of the"first"part of the"bathtub"model, i.e.r(t) is non-increasing for t<r and is constant for t> r. Asymptotic distribution of one of the estimates proposed earlier has been investigated in this paper. This leads to a test for the hypothesis HQ r<r ₀ vs H :r>r (where TQ > 0). Asymptotic expression for the power of this test under Pitman alternatives is derived. Some simulations are reported.