The optimal Lp norm estimator in linear regression models

The least squares estimator is usually applied when estimating the parameters in linear regression models. As this estimator is sensitive to departures from normality in the residual distribution, several alternatives have been proposed. The L_p norm estimators is one class of such alternatives. It has been proposed that the kurtosis of the residual distribution be taken into account when a choice of estimator in the L_p norm class is made (i.e. the choice of p). In this paper, the asymtotic variance of the estimators is used as the criterion in the choice of p. It is shown that when this criterion is applied, other characteristics of the residual distribution than the kurtosis (namely moments of order p-2 and 2p-2) are important.     

Consistency of l 1 estimates in censored linear regression models

In this paper, the regression model with a nonnegativity constraint on the dependent variable is considered. Under weak conditions, L ₁ estimates of the regression coefficients are shown to be consistent.     

Partially linear censored quantile regression

Censored regression quantile (CRQ) methods provide a powerful and flexible approach to the analysis of censored survival data when standard linear models are felt to be appropriate. In many cases however, greater flexibility is desired to go beyond the usual multiple regression paradigm. One area of common interest is that of partially linear models: one (or more) of the explanatory covariates are assumed to act on the response through a non-linear function. Here the CRQ approach of Portnoy (J Am Stat Assoc 98:1001–1012, 2003) is extended to this partially linear setting. Basic consistency results are presented. A simulation experiment and unemployment example justify the value of the partially linear approach over methods based on the Cox proportional hazards model and on methods not permitting nonlinearity.     

Added variable plots for linear regression with censored data

In linear regression the structure of the hat matrix plays an important part in regression diagnostics. In this note we investigate the properties of the hat matrix for regression with censored responses in the presence of one or more explanatory variables observed without censoring. The censored points in the scatterplot are renovated to positions had they been observed without censoring in a renovation process based on Buckley-James censored regression estimators. This allows natural links to be established with the structure of ordinary least squares estimators. In particular, we show that the renovated hat matrix may be partitioned in a manner which assists in deciding whether further explanatory variables should be added to the linear model. The added variable plot for regression with censored data is developed as a diagnostic tool for this decision process.     

A large-scale monte carlo study of the buckley-james estimator with censored data

Estimation in the presence of censoring is an important problem. In the linear model, the Buckley-James method proceeds iteratively by estimating the censored values than re-estimating the regression coeffi- cients. A large-scale Monte Carlo simulation technique has been developed to test the performance of the Buckley-James (denoted B-J) estimator. One hundred and seventy two randomly generated data sets, each with three thousand replications, based on four failure distributions, four censoring patterns, three sample sizes and four censoring rates have been investigated, and the results are presented. It is found that, except for Type I1 censoring, the B-J estimator is essentially unbiased, even when the data sets with small sample sizes are subjected to a high censoring rate. The variance formula suggested by Buckley and James (1979) is shown to be sensitive to the failure distribution. If the censoring rate is kept constant along the covariate line, the sample variance of the estimator appears to be insensitive to the censoring pattern with a selected failure distribution. Oscillation of the convergence values associated with the B-J estimator is illustrated and thoroughly discussed.     

Laplace approximations for censored linear regression models

We consider approximate Bayesian inference about scalar parameters of linear regression models with possible censoring. A second-order expansion of their Laplace posterior is seen to have a simple and intuitive form for logconcave error densities with nondecreasing hazard functions. The accuracy of the approximations is assessed for normal and Gumbel errors when the number of regressors increases with sample size. Perturbations of the prior and the likelihood are seen to be easily accommodated within our framework. Links with the work of DiCiccio et al. (1990) and Viveros and Sprott (1987) extend the applicability of our results to conditional frequentist inference based on likelihood-ratio statistics.     

Weak and strong uniform consistency rates of kernel density estimates for randomly censored data

Let X₁,., X_n, be i.i.d. random variables with distribution function F, and let Y₁,.,.,Y_n be i.i.d. with distribution function G. For i = 1, 2,.,., n set δ_i, = 1 if X_i ≤ Y_i, and 0 otherwise, and X_i, = min{X_i, K_i}. A kernel-type density estimate of f, the density function of F w.r.t. Lebesgue measure on the Borel o-field, based on the censored data (δ_i, X_i), i = 1,.,.,n, is considered. Weak and strong uniform consistency properties over the whole real line are studied. Rates of convergence results are established under higher-order differentiability assumption on f. A procedure for relaxing such assumptions is also proposed.     

On the weak convergence of kernel density estimators in Lp spaces

Since its introduction, the pointwise asymptotic properties of the kernel estimator f?_n of a probability density function f on ?^d, as well as the asymptotic behaviour of its integrated errors, have been studied in great detail. Its weak convergence in functional spaces, however, is a more difficult problem. In this paper, we show that if f_n(x)=(f?_n(x)) and (r_n) is any nonrandom sequence of positive real numbers such that r_n/√n→0 then if r_n(f?_n?f_n) converges to a Borel measurable weak limit in a weighted L^p space on ?^d, with 1≤p<∞, the limit must be 0. We also provide simple conditions for proving or disproving the existence of this Borel measurable weak limit.     

Convergence rate of wavelet density estimator with data missing randomly when covariables are present

In this article, we study global L₂ error of non linear wavelet estimator of density in the Besov space B^s_pq for missing data model when covariables are present and prove that the estimator can achieve the optimal rate of convergence, which is similar to the result studied by Donoho et al. (1996) Donoho, D.L., Johnstone, I.M., Kerkyacharian, G., Picard, D. (1996). Density estimation by wavelet thresholding. Ann. Stat. 24:508–539.[Crossref], [Web of Science ®] , [Google Scholar] in complete independent data case with term-by-term thresholding of the empirical wavelet coefficients. Finite-sample behavior of the proposed estimator is explored via simulations.     

Least absolute value regression: a special case of piecewise linear minimization

The Barrodale and Roberts algorithm for least absolute value (LAV) regression and the algorithm proposed by Bartels and Conn both have the advantage that they are often able to skip across points at which the conventional simplex-method algorithms for LAV regression would be required to carry out an (expensive) pivot operation.

We indicate here that this advantage holds in the Bartels-Conn approach for a wider class of problems: the minimization of piecewise linear functions. We show how LAV regression, restricted LAV regression, general linear programming and least maximum absolute value regression can all be easily expressed as piecewise linear minimization problems.     

Variable selection in linear regression based on ridge estimator

In this article, we consider the problem of variable selection in linear regression when multicollinearity is present in the data. It is well known that in the presence of multicollinearity, performance of least square (LS) estimator of regression parameters is not satisfactory. Consequently, subset selection methods, such as Mallow's Cp, which are based on LS estimates lead to selection of inadequate subsets. To overcome the problem of multicollinearity in subset selection, a new subset selection algorithm based on the ridge estimator is proposed. It is shown that the new algorithm is a better alternative to Mallow's Cp when the data exhibit multicollinearity.     

Non parametric regression estimations over Lp risk based on biased data

Using a wavelet basis, Chesneau and Shirazi study the estimation of one-dimensional regression functions in a biased non parametric model over L² risk (see Chesneau, C and Shirazi, E. Non parametric wavelet regression based on biased data, Communication in Statistics – Theory and Methods, 43: 2642–2658, 2014). This article considers d-dimensional regression function estimation over L^p?(1 ? p < ∞) risk. It turns out that our results reduce to the corresponding theorems of Chesneau and Shirazi’s theorems, when d = 1 and p = 2.     

Performance of Wald-type estimator for parametric component in partial linear regression with a mixture of Berkson and classical error models

This article discusses a consistent and almost unbiased estimation approach in partial linear regression for parameters of interest when the regressors are contaminated with a mixture of Berkson and classical errors. Advantages of the presented procedure are: (1) random errors and observations are not necessarily to be parametric settings; (2) there is no need to use additional sample information, and to consider the estimation of nuisance parameters. We will examine the performance of our presented estimate in a variety of numerical examples through Monte Carlo simulation. The proposed approach is also illustrated in the analysis of an air pollution data.     

Asymptotics for L2-norm of ARCH innovation density estimator

In this paper we consider the asymptotic properties of the ARCH innovation density estimator. We obtain the asymptotic normality of the Bickel-Rosenblatt test statistic (based on our density estimator) under the null hypothesis, which is the same as in the case of the one sample set up (given in Bickel and Rosenblatt, 1973). We also show the strong consistency of the estimator for the true density in L2-norm.     

L1 for the simple linear regression model

By modifying the direct method to solve the overdetermined linear system we are able to present an algorithm for L₁ estimation which appears to be superior computationally to any other known algorithm for the simple linear regression problem.     

Nonparametric density estimation for censored survival data: Regression-spline approach

A method for nonparametric estimation of density based on a randomly censored sample is presented. The density is expressed as a linear combination of cubic M -splines, and the coefficients are determined by pseudo-maximum-likelihood estimation (likelihood is maximized conditionally on data-dependent knots). By using regression splines (small number of knots) it is possible to reduce the estimation problem to a space of low dimension while preserving flexibility, thus striking a compromise between parametric approaches and ordinary nonparametric approaches based on spline smoothing. The number of knots is determined by the minimum AIC. Examples of simulated and real data are presented. Asymptotic theory and the bootstrap indicate that the precision and the accuracy of the estimates are satisfactory.