Adaptive Estimation Using Weighted Least Squares

This paper proposes an adaptive estimator that is more precise than the ordinary least squares estimator if the distribution of random errors is skewed or has long tails. The adaptive estimates are computed using a weighted least squares approach with weights based on the lengths of the tails of the distribution of residuals. Smaller weights are assigned to those observations that have residuals in the tails of long-tailed distributions and larger weights are assigned to observations having residuals in the tails of short-tailed distributions. Monte Carlo methods are used to compare the performance of the proposed estimator and the performance of the ordinary least squares estimator. The estimates that were studied in this simulation include the difference between the means of two populations, the mean of a symmetric distribution, and the slope of a regression line. The adaptive estimators are shown to have lower mean squared errors than those for the ordinary least squares estimators for short-tailed, long-tailed, and skewed distributions, provided the sample size is at least 20. The ordinary least squares estimator has slightly lower mean squared error for normally distributed errors. The adaptive estimator is recommended for general use for studies having sample sizes of at least 20 observations unless the random errors are known to be normally distributed.     

A new extended generalized Gompertz distribution with statistical properties and simulations

Abstract

Statistical distributions are very useful in describing and predicting real world phenomena. In many applied areas there is a clear need for the extended forms of the well-known distributions. Generally, the new distributions are more flexible to model real data that present a high degree of skewness and kurtosis. The choice of the best-suited statistical distribution for modeling data is very important.

In this article, we proposed an extended generalized Gompertz (EGGo) family of EGGo. Certain statistical properties of EGGo family including distribution shapes, hazard function, skewness, limit behavior, moments and order statistics are discussed. The flexibility of this family is assessed by its application to real data sets and comparison with other competing distributions. The maximum likelihood equations for estimating the parameters based on real data are given. The performances of the estimators such as maximum likelihood estimators, least squares estimators, weighted least squares estimators, Cramer-von-Mises estimators, Anderson-Darling estimators and right tailed Anderson-Darling estimators are discussed. The likelihood ratio test is derived to illustrate that the EGGo distribution is better than other nested models in fitting data set or not. We use R software for simulation in order to perform applications and test the validity of this model.     

Rank estimation for the functional linear model

This article discusses the estimation of the parameter function for a functional linear regression model under heavy-tailed errors' distributions and in the presence of outliers. Standard approaches of reducing the high dimensionality, which is inherent in functional data, are considered. After reducing the functional model to a standard multiple linear regression model, a weighted rank-based procedure is carried out to estimate the regression parameters. A Monte Carlo simulation and a real-world example are used to show the performance of the proposed estimator and a comparison made with the least-squares and least absolute deviation estimators.     

Exponentiated Teissier distribution with increasing,decreasing and bathtub hazard functions

This article introduces a two-parameter exponentiated Teissier distribution. It is the main advantage of the distribution to have increasing, decreasing and bathtub shapes for its hazard rate function. The expressions of the ordinary moments, identifiability, quantiles, moments of order statistics, mean residual life function and entropy measure are derived. The skewness and kurtosis of the distribution are explored using the quantiles. In order to study two independent random variables, stress–strength reliability and stochastic orderings are discussed. Estimators based on likelihood, least squares, weighted least squares and product spacings are constructed for estimating the unknown parameters of the distribution. An algorithm is presented for random sample generation from the distribution. Simulation experiments are conducted to compare the performances of the considered estimators of the parameters and percentiles. Three sets of real data are fitted by using the proposed distribution over the competing distributions.     

Parameter estimation for generalized random coefficient autoregressive processes

A generalized random coefficient autoregressive (GRCA) process is introduced in which the random coefficients are permitted to be correlated with the error process. The ordinary random coefficient autoregressive process, the Markovian bilinear model and its generalization, and the random coefficient exponential autoregressive process, among others, are seen to be special cases of the GRCA process. Conditional least squares, and weighted least-squares estimators of the mean of the random coefficient vector are derived and their limit distributions are studied. Estimators of the variance-covariance parameters are also discussed. A simulation study is presented which shows that the weighted least-squares estimator dominates the unweighted least-squares estimator.     

Topp–Leone normal distribution with application to increasing failure rate data

In this article, we proposed a new three-parameter probability distribution, called Topp–Leone normal, for modelling increasing failure rate data. The distribution is obtained by using Topp–Leone-X family of distributions with normal as a baseline model. The basic properties including moments, quantile function, stochastic ordering and order statistics are derived here. The estimation of unknown parameters is approached by the method of maximum likelihood, least squares, weighted least squares and maximum product spacings. An extensive simulation study is carried out to compare the long-run performance of the estimators. Applicability of the distribution is illustrated by means of three real data analyses over existing distributions.     

Local Linear Regression in Proportional Hazards Model with Censored Data

In this article we study the method of nonparametric regression based on a transformation model, under which an unknown transformation of the survival time is nonlinearly, even more, nonparametrically, related to the covariates with various error distributions, which are parametrically specified with unknown parameters. Local linear approximations and locally weighted least squares are applied to obtain estimators for the effects of covariates with censored observations. We show that the estimators are consistent and asymptotically normal. This transformation model, coupled with local linear approximation techniques, provides many alternatives to the more general proportional hazards models with nonparametric covariates.     

Modified maximum likelihood estimator under the Jones and Faddy's skew t-error distribution for censored regression model

It is well-known that classical Tobit estimator of the parameters of the censored regression (CR) model is inefficient in case of non-normal error terms. In this paper, we propose to use the modified maximum likelihood (MML) estimator under the Jones and Faddy''s skew t-error distribution, which covers a wide range of skew and symmetric distributions, for the CR model. The MML estimators, providing an alternative to the Tobit estimator, are explicitly expressed and they are asymptotically equivalent to the maximum likelihood estimator. A simulation study is conducted to compare the efficiencies of the MML estimators with the classical estimators such as the ordinary least squares, Tobit, censored least absolute deviations and symmetrically trimmed least squares estimators. The results of the simulation study show that the MML estimators work well among the others with respect to the root mean square error criterion for the CR model. A real life example is also provided to show the suitability of the MML methodology.     

On LSE in regression model for long-range dependent random fields on spheres

We study the asymptotic behaviour of least squares estimators (LSE) in regression models for long-range dependent random fields observed on spheres. The LSE can be given as a weighted functional of long-range dependent random fields. It is known that in this scenario the limits can be non-Gaussian. We derive the limit distribution and the corresponding rate of convergence for the estimators. The results were obtained under rather general assumptions on the random fields. Simulation studies were conducted to support theoretical findings.     

Change points in nonparametric regresion functions

Estimators of location and size of jumps or discontinuities in a regression function and/or its derivatives are proposed. The estimators are based on the analysis of residuals obtained from the locally weighted least squares regression. The proposed estimators adapt to both fixed and random designs. The asymptotic properties of the estimators are investigated. The method is illustrated through simulation studies.     

Multiple Linear Regression Model Under Nonnormality

Abstract

We consider multiple linear regression models under nonnormality. We derive modified maximum likelihood estimators (MMLEs) of the parameters and show that they are efficient and robust. We show that the least squares esimators are considerably less efficient. We compare the efficiencies of the MMLEs and the M estimators for symmetric distributions and show that, for plausible alternatives to an assumed distribution, the former are more efficient. We provide real-life examples.     

Inference under Heteroscedasticity of Unknown Form Using an Adaptive Estimator

The heteroscedasticity consistent covariance matrix estimators are commonly used for the testing of regression coefficients when error terms of regression model are heteroscedastic. These estimators are based on the residuals obtained from the method of ordinary least squares and this method yields inefficient estimators in the presence of heteroscedasticity. It is usual practice to use estimated weighted least squares method or some adaptive methods to find efficient estimates of the regression parameters when the form of heteroscedasticity is unknown. But HCCM estimators are seldom derived from such efficient estimators for testing purposes in the available literature. The current article addresses the same concern and presents the weighted versions of HCCM estimators. Our numerical work uncovers the performance of these estimators and their finite sample properties in terms of interval estimation and null rejection rate.     

Penalized weighted composite quantile regression in the linear regression model with heavy-tailed autocorrelated errors

In this paper, a penalized weighted composite quantile regression estimation procedure is proposed to estimate unknown regression parameters and autoregression coefficients in the linear regression model with heavy-tailed autoregressive errors. Under some conditions, we show that the proposed estimator possesses the oracle properties. In addition, we introduce an iterative algorithm to achieve the proposed optimization problem, and use a data-driven method to choose the tuning parameters. Simulation studies demonstrate that the proposed new estimation method is robust and works much better than the least squares based method when there are outliers in the dataset or the autoregressive error distribution follows heavy-tailed distributions. Moreover, the proposed estimator works comparably to the least squares based estimator when there are no outliers and the error is normal. Finally, we apply the proposed methodology to analyze the electricity demand dataset.     

Regression with an infinite number of observations applied to estimating the parameters of the stable distribution using the empirical characteristic function

ABSTRACT

The parameters of stable law parameters can be estimated using a regression based approach involving the empirical characteristic function. One approach is to use a fixed number of points for all parameters of the distribution to estimate the characteristic function. In this work the results are derived where all points in an interval is used to estimate the empirical characteristic function, thus least squares estimators of a linear function of the parameters, using an infinite number of observations. It was found that the procedure performs very good in small samples.     

Estimation of parameters in heavy-tailed distribution when its second order tail parameter is known

Estimating parameters in heavy-tailed distribution plays a central role in extreme value theory. It is well known that classical estimators based on the first order asymptotics such as the Hill, rank-based and QQ estimators are seriously biased under finer second order regular variation framework. To reduce the bias, many authors proposed the so-called second order reduced bias estimators for both first and second order tail parameters. In this work, estimation of parameters in heavy-tailed distributions are studied under the second order regular variation framework when the second order parameter in the distribution tail is known. This is motivated in large part by a recent work by the authors showing that the second order tail parameter is known for a large class of popular random difference equations (for example, ARCH models). The focus is on least squares estimators that generalize rank-based and QQ estimators. Though other possible estimators are also briefly discussed, the least squares estimators are most simple to use and perform best for finite samples in Monte Carlo simulations.     

Local Linear Least Squares Kernel Regression for Linear and Circular Predictors

We extend nonparametric regression models with local linear least squares fitting using kernel weights to the case of linear and circular predictors. We derive the asymptotic properties of the conditional bias and variance of bivariate local linear least squares kernel estimators. A small simulation study and a real experiment are given.     

On shrinkage least squares estimation in a parallelism problem

In a multi-sample simple regression model, generally, homogeneity of the regression slopes leads to improved estimation of the intercepts. Analogous to the preliminary test estimators, (smooth) shrinkage least squares estimators of Intercepts based on the James-Stein rule on regression slopes are considered. Relative pictures on the (asymptotic) risk of the classical, preliminary test and the shrinkage least squares estimators are also presented. None of the preliminary test and shrinkage least squares estimators may dominate over the other, though each of them fares well relative to the other estimators.     

Robust 2k factorial design with Weibull error distributions

It is well known that the least squares method is optimal only if the error distributions are normally distributed. However, in practice, non-normal distributions are more prevalent. If the error terms have a non-normal distribution, then the efficiency of least squares estimates and tests is very low. In this paper, we consider the 2^k factorial design when the distribution of error terms are Weibull W(p,σ). From the methodology of modified likelihood, we develop robust and efficient estimators for the parameters in 2^k factorial design. F statistics based on modified maximum likelihood estimators (MMLE) for testing the main effects and interaction are defined. They are shown to have high powers and better robustness properties as compared to the normal theory solutions. A real data set is analysed.     

A Monte Carlo simulation study on partially adaptive estimators of linear regression models

This paper presents a comprehensive comparison of well-known partially adaptive estimators (PAEs) in terms of efficiency in estimating regression parameters. The aim is to identify the best estimators of regression parameters when error terms follow from normal, Laplace, Student's t, normal mixture, lognormal and gamma distribution via the Monte Carlo simulation. In the results of the simulation, efficient PAEs are determined in the case of symmetric leptokurtic and skewed leptokurtic regression error data. Additionally, these estimators are also compared in terms of regression applications. Regarding these applications, using certain standard error estimators, it is shown that PAEs can reduce the standard error of the slope parameter estimate relative to ordinary least squares.     

Regression model with elliptically contoured errors

For the regression model y=X β+ε where the errors follow the elliptically contoured distribution, we consider the least squares, restricted least squares, preliminary test, Stein-type shrinkage and positive-rule shrinkage estimators for the regression parameters, β.

We compare the quadratic risks of the estimators to determine the relative dominance properties of the five estimators.