Estimating Percentiles of Time-to-Failure Distribution Obtained from a Linear Degradation Model Using Kernel Density Method

In this article, we propose a nonparametric estimator for percentiles of the time-to-failure distribution obtained from a linear degradation model using the kernel density method. The properties of the proposed kernel estimator are investigated and compared with well-known maximum likelihood and ordinary least squares estimators via a simulation technique. The mean squared error and the length of the bootstrap confidence interval are used as the basis criteria of the comparisons. The simulation study shows that the performance of the kernel estimator is acceptable as a general estimator. When the distribution of the data is assumed to be known, the maximum likelihood and ordinary least squares estimators perform better than the kernel estimator, while the kernel estimator is superior when the assumption of our knowledge of the data distribution is violated. A comparison among different estimators is achieved using a real data set.     

Kernel Density-Based Linear Regression Estimate

For linear regression models with non normally distributed errors, the least squares estimate (LSE) will lose some efficiency compared to the maximum likelihood estimate (MLE). In this article, we propose a kernel density-based regression estimate (KDRE) that is adaptive to the unknown error distribution. The key idea is to approximate the likelihood function by using a nonparametric kernel density estimate of the error density based on some initial parameter estimate. The proposed estimate is shown to be asymptotically as efficient as the oracle MLE which assumes the error density were known. In addition, we propose an EM type algorithm to maximize the estimated likelihood function and show that the KDRE can be considered as an iterated weighted least squares estimate, which provides us some insights on the adaptiveness of KDRE to the unknown error distribution. Our Monte Carlo simulation studies show that, while comparable to the traditional LSE for normal errors, the proposed estimation procedure can have substantial efficiency gain for non normal errors. Moreover, the efficiency gain can be achieved even for a small sample size.     

Minimax Linear Smoothers

We consider the problem of estimating the mean of a multivariate distribution. As a general alternative to penalized least squares estimators, we consider minimax estimators for squared error over a restricted parameter space where the restriction is determined by the penalization term. For a quadratic penalty term, the minimax estimator among linear estimators can be found explicitly. It is shown that all symmetric linear smoothers with eigenvalues in the unit interval can be characterized as minimax linear estimators over a certain parameter space where the bias is bounded. The minimax linear estimator depends on smoothing parameters that must be estimated in practice. Using results in Kneip (1994), this can be done using Mallows' C _L -statistic and the resulting adaptive estimator is now asymptotically minimax linear. The minimax estimator is compared to the penalized least squares estimator both in finite samples and asymptotically.     

Local Linear Regression in Proportional Hazards Model with Censored Data

In this article we study the method of nonparametric regression based on a transformation model, under which an unknown transformation of the survival time is nonlinearly, even more, nonparametrically, related to the covariates with various error distributions, which are parametrically specified with unknown parameters. Local linear approximations and locally weighted least squares are applied to obtain estimators for the effects of covariates with censored observations. We show that the estimators are consistent and asymptotically normal. This transformation model, coupled with local linear approximation techniques, provides many alternatives to the more general proportional hazards models with nonparametric covariates.     

Linear least squares estimates and nonlinear means

The consistency and asymptotic normality of a linear least squares estimate of the form (X'X)^-X'Y when the mean is not Xβ is investigated in this paper. The least squares estimate is a consistent estimate of the best linear approximation of the true mean function for the design chosen. The asymptotic normality of the least squares estimate depends on the design and the asymptotic mean may not be the best linear approximation of the true mean function. Choices of designs which allow large sample inferences to be made about the best linear approximation of the true mean function are discussed.     

Estimation in Singular Linear Models with Stochastic Linear Restrictions and Linear Equality Restrictions

This article is concerned with the parameter estimation in a singular linear regression model with stochastic linear restrictions and linear equality restrictions simultaneously. A new estimator is introduced and it is proved that the proposed estimator is superior to the least squares estimator and singular mixed estimator in the mean squared error sense under certain conditions.     

Two Kinds of Restricted Estimators in Singular Linear Regression

In this article, the parameter estimators in singular linear model with linear equality restrictions are considered. The restricted root estimator and the generalized restricted root estimator are proposed and some properties of the estimators are also studied. Furthermore, we compare them with the restricted unified least squares estimator and show their sufficient conditions under which their superior over the restricted unified least squares estimator in terms of mean squares error, and discuss the choice of the unknown parameters of the generalized restricted root estimator.     

Nonparametric density estimation for multivariate bounded data

We propose a new nonparametric estimator for the density function of multivariate bounded data. As frequently observed in practice, the variables may be partially bounded (e.g. nonnegative) or completely bounded (e.g. in the unit interval). In addition, the variables may have a point mass. We reduce the conditions on the underlying density to a minimum by proposing a nonparametric approach. By using a gamma, a beta, or a local linear kernel (also called boundary kernels), in a product kernel, the suggested estimator becomes simple in implementation and robust to the well known boundary bias problem. We investigate the mean integrated squared error properties, including the rate of convergence, uniform strong consistency and asymptotic normality. We establish consistency of the least squares cross-validation method to select optimal bandwidth parameters. A detailed simulation study investigates the performance of the estimators. Applications using lottery and corporate finance data are provided.     

Statistical Analysis of Non Linear Least Squares Estimation for Harmonic Signals in Multiplicative and Additive Noise

Abstract

This article concerns the stochastically constrained linear model under a biased assumption. We propose a quasi-stochastically constrained least squares estimator. Furthermore, we provide the expectation of this estimator, demonstrate its consistency and asymptotic normality. In the end of the article, the simulation study of the new estimator shows that it is superior to the least squares estimator, ridge estimator, and the linear constrained estimators under certain conditions by comparing the mean squared errors of these estimators.     

Local and Variable Bandwidths and Local Linear Regression

Two types of non-global bandwidth, which may be called local and variable, have been defined in attempts to improve the performance of kernel density estimators. In nonparametric regression, local linear fitting has become a method of much popularity. It is natural, therefore, to consider the use of non-global bandwidths in the local linear context, and indeed local bandwidths are often used. In this paper, it is observed that a natural proposal in the literature for combining variable bandwidths with local linear fitting fails in the sense that the resulting mean squared error properties are those normally associated with local rather than variable bandwidths. We are able to understand why this happens in terms of weightings that are involved. We also attempt to investigate how the bias reduction expected of well-chosen variable bandwidths might be achieved in conjunction with local linear fitting.     

Variable Selection for Partially Linear Models with Randomly Censored Data

This article proposes a variable selection procedure for partially linear models with right-censored data via penalized least squares. We apply the SCAD penalty to select significant variables and estimate unknown parameters simultaneously. The sampling properties for the proposed procedure are investigated. The rate of convergence and the asymptotic normality of the proposed estimators are established. Furthermore, the SCAD-penalized estimators of the nonzero coefficients are shown to have the asymptotic oracle property. In addition, an iterative algorithm is proposed to find the solution of the penalized least squares. Simulation studies are conducted to examine the finite sample performance of the proposed method.     

Multiple Linear Regression Model Under Nonnormality

Abstract

We consider multiple linear regression models under nonnormality. We derive modified maximum likelihood estimators (MMLEs) of the parameters and show that they are efficient and robust. We show that the least squares esimators are considerably less efficient. We compare the efficiencies of the MMLEs and the M estimators for symmetric distributions and show that, for plausible alternatives to an assumed distribution, the former are more efficient. We provide real-life examples.     

Ridge Estimation to the Restricted Linear Model

This article is concerned with the problem of multicollinearity in a linear model with linear restrictions. After introducing a spheral restricted condition, a new restricted ridge estimation method is proposed by minimizing the sum of squared residuals. The property of the new estimator in its superiority over the ordinary restricted least squares estimation is then theoretically analyzed. Furthermore, a sufficient and necessary condition for selecting the ridge parameter k is obtained. To simplify the selection of the ridge parameter, a sufficient condition is also given. Finally, a numerical example demonstrates the merit of the new method in the aspect of solving the multicollinearity over the ordinary restricted least squares estimation.     

On Necessary and Sufficient Conditions for Ordinary Least Squares Estimators to Be Best Linear Unbiased Estimators

Two often-quoted necessary and sufficient conditions for ordinary least squares estimators to be best linear unbiased estimators are described. Another necessary and sufficient condition is described, providing an additional tool for checking to see whether the covariance matrix of a given linear model is such that the ordinary least squares estimator is also the best linear unbiased estimator. The new condition is used to show that one of the two published conditions is only a sufficient condition.     

Boundary and Bias Correction in Kernel Hazard Estimation

A new class of local linear hazard estimators based on weighted least square kernel estimation is considered. The class includes the kernel hazard estimator of Ramlau-Hansen (1983), which has the same boundary correction property as the local linear regression estimator (see Fan & Gijbels, 1996). It is shown that all the local linear estimators in the class have the same pointwise asymptotic properties. We derive the multiplicative bias correction of the local linear estimator. In addition we propose a new bias correction technique based on bootstrap estimation of additive bias. This latter method has excellent theoretical properties. Based on an extensive simulation study where we compare the performance of competing estimators, we also recommend the use of the additive bias correction in applied work.     

Testing the adequacy of a linear model VIA critical smoothing

An important problem for fitting local linear regression is the choice of the smoothing parameter. As the smoothing parameter becomes large, the estimator tends to a straight line, which is the least squares fit in the ordinary linear regression setting. This property may be used to assess the adequacy of a simple linear model. Motivated by Silverman's (1981) work in kernel density estimation, a suitable test statistic is the critical smoothing parameter where the estimate changes from nonlinear to linear, while linearity or non- linearity requires a more precise judgment. We define the critical smoothing parameter through the approximate F-tests by Hastie and Tibshirani (1990). To assess the significance, the “wild bootstrap” procedure is used to replicate the data and the proportion of bootstrap samples which give a nonlinear estimate when using the critical bandwidth is obtained as the p-value. Simulation results show that the critical smoothing test is useful in detecting a wide range of alternatives.     

Outlier Resistant Estimation in Difference-Based Semiparametric Partially Linear Models

Partially linear models are extensions of linear models that include a nonparametric function of some covariate allowing an adequate and more flexible handling of explanatory variables than in linear models. The difference-based estimation in partially linear models is an approach designed to estimate parametric component by using the ordinary least squares estimator after removing the nonparametric component from the model by differencing. However, it is known that least squares estimates do not provide useful information for the majority of data when the error distribution is not normal, particularly when the errors are heavy-tailed and when outliers are present in the dataset. This paper aims to find an outlier-resistant fit that represents the information in the majority of the data by robustly estimating the parametric and the nonparametric components of the partially linear model. Simulations and a real data example are used to illustrate the feasibility of the proposed methodology and to compare it with the classical difference-based estimator when outliers exist.     

Testing Covariate Effects in Aalen's Linear Hazard Model

The performance of tests in Aalen's linear regression model is studied using asymptotic power calculations and stochastic simulation. Aalen's original least squares test is compared to two modifications: a weighted least squares test with correct weights and a test where the variance is re-estimated under the null hypothesis. The test with re-estimated variance provides the highest power of the tests for the setting of this paper, and the gain is substantial for covariates following a skewed distribution like the exponential. It is further shown that Aalen's choice for weight function with re-estimated variance is optimal in the one-parameter case against proportional alternatives.     

Constructive Cross-Validation in Linear Prediction

Approaches for regressor construction in the linear prediction problem are investigated in a framework similar to partial least squares and continuum regression, but weighted to allow for custom specification of an evaluative scheme. A cross-validatory continuum regression procedure is proposed, and shown to compare well with ordinary continuum regression in empirical demonstrations.     

Series Estimation in Partially Linear In‐Slide Regression Models

Abstract. The partially linear in‐slide model (PLIM) is a useful tool to make econometric analyses and to normalize microarray data. In this article, by using series approximations and a least squares procedure, we propose a semiparametric least squares estimator (SLSE) for the parametric component and a series estimator for the non‐parametric component. Under weaker conditions than those imposed in the literature, we show that the SLSE is asymptotically normal and that the series estimator attains the optimal convergence rate of non‐parametric regression. We also investigate the estimating problem of the error variance. In addition, we propose a wild block bootstrap‐based test for the form of the non‐parametric component. Some simulation studies are conducted to illustrate the finite sample performance of the proposed procedure. An example of application on a set of economical data is also illustrated.