Efficient estimation of the PDF and the CDF of the inverse Rayleigh distribution

In this paper, we consider the estimation of the probability density function and the cumulative distribution function of the inverse Rayleigh distribution. In this regard, the following estimators are considered: uniformly minimum variance unbiased estimator, maximum likelihood (ML) estimator, percentile estimator, least squares estimator and weighted least squares estimator. To do so, analytical expressions are derived for the mean integrated squared error. As the result of simulation studies and real data applications indicate, when the sample size is not very small the ML estimator performs better than the others.     

Adaptive Estimation Using Weighted Least Squares

This paper proposes an adaptive estimator that is more precise than the ordinary least squares estimator if the distribution of random errors is skewed or has long tails. The adaptive estimates are computed using a weighted least squares approach with weights based on the lengths of the tails of the distribution of residuals. Smaller weights are assigned to those observations that have residuals in the tails of long-tailed distributions and larger weights are assigned to observations having residuals in the tails of short-tailed distributions. Monte Carlo methods are used to compare the performance of the proposed estimator and the performance of the ordinary least squares estimator. The estimates that were studied in this simulation include the difference between the means of two populations, the mean of a symmetric distribution, and the slope of a regression line. The adaptive estimators are shown to have lower mean squared errors than those for the ordinary least squares estimators for short-tailed, long-tailed, and skewed distributions, provided the sample size is at least 20. The ordinary least squares estimator has slightly lower mean squared error for normally distributed errors. The adaptive estimator is recommended for general use for studies having sample sizes of at least 20 observations unless the random errors are known to be normally distributed.     

Estimating Percentiles of Time-to-Failure Distribution Obtained from a Linear Degradation Model Using Kernel Density Method

In this article, we propose a nonparametric estimator for percentiles of the time-to-failure distribution obtained from a linear degradation model using the kernel density method. The properties of the proposed kernel estimator are investigated and compared with well-known maximum likelihood and ordinary least squares estimators via a simulation technique. The mean squared error and the length of the bootstrap confidence interval are used as the basis criteria of the comparisons. The simulation study shows that the performance of the kernel estimator is acceptable as a general estimator. When the distribution of the data is assumed to be known, the maximum likelihood and ordinary least squares estimators perform better than the kernel estimator, while the kernel estimator is superior when the assumption of our knowledge of the data distribution is violated. A comparison among different estimators is achieved using a real data set.     

Numerical computation of exact moments of the least squares estimator in a first-order stationary autoregressive model

This paper examines the least squares estimator of the autoregressive coefficient in a first-order sta¬tionary autoregressive model. Exact lower-order moments are computed by numerical integrations. From the study of moment values, it is found that the exact distribution of the least squares estimator may be well approximated by a beta distribution.     

Estimation of parameters in a generalized GMANOVA model based on an outer product analogy and least squares

This paper investigates estimation of parameters in a combination of the multivariate linear model and growth curve model, called a generalized GMANOVA model. Making analogy between the outer product of data vectors and covariance yields an approach to directly do least squares to covariance. An outer product least squares estimator of covariance (COPLS estimator) is obtained and its distribution is presented if a normal assumption is imposed on the error matrix. Based on the COPLS estimator, two-stage generalized least squares estimators of the regression coefficients are derived. In addition, asymptotic normalities of these estimators are investigated. Simulation studies have shown that the COPLS estimator and two-stage GLS estimators are alternative competitors with more efficiency in the sense of sample mean, standard deviations and mean of the variance estimates to the existing ML estimator in finite samples. An example of application is also illustrated.     

A likelihood based estimator for vector autoregressive processes

A onestep estimator, which is an approximation to the unconditional maximum likelihood estimator (MLE) of the coefficient matrices of a Gaussian vector autoregressive process is presented. The onestep estimator is easy to compute and can be computed using standard software. Unlike the computation of the unconditional MLE, the computation of the onestep estimator does not require any iterative optimization and the computation is numerically stable. In finite samples the onestep estimator generally has smaller mean square error than the ordinary least squares estimator. In a simple model, where the unconditional MLE can be computed, numerical investigation shows that the onestep estimator is slightly worse than the unconditional MLE in terms of mean square error but superior to the ordinary least squares estimator. The limiting distribution of the onestep estimator for processes with some unit roots is derived.     

On weak consistency in linear models with equi-correlated random errors

A new estimator in linear models with equi-correlated random errors is postulated. Consistency properties of the proposed estimator and the ordinary least squares estimator are studied. It is shown that the new estimator has smaller variance than the usual least squares estimator under some mild conditions. In addition, it is observed that the new estimator tends to be weakly consistent in many cases where the usual least squares estimator is not.     

Estimation of Weibull parameters from common percentiles

Estimation of Weibull distribution shape and scale parameters is accomplished through use of symmetrically located percentiles from a sample. The process requires algebraic solution of two equations derived from the cumulative distribution function. Three alternatives examined are compared for precision and variability with maximum likelihood (MLE) and least squares (LS) estimators. The best percentile estimator (using the 10th and 90th) is inferior to MLE in variability and to one least squares estimator in accuracy and variability to a small degree. However, application of a correction factor related to sample size improves the percentile estimator substantially, making it more accurate than LS.     

On generalized least squares estimation of the weibull distribution

A simple estimation procedure, based on the generalized least squares method, for the parameters of the Weibull distribution is described and investigated. Through a simulation study, this estimation technique is compared with maximum likelihood estimation, ordinary least squares estimation, and Menon's estimation procedure; this comparison is based on observed relative efficiencies (that is, the ratio of the Cramer-Rao lower bound to the observed mean squared error). Simulation results are presented for samples of size 25. Among the estimators considered in this simulation study, the generalized least squares estimator was found to be the "best" estimator for the shape parameter and a close competitor to the maximum likelihood estimator of the scale parameter.     

Efficient Estimation of the PDF and the CDF of the Weibull Extension Model

The Weibull extension model is a useful extension of the Weibull distribution, allowing for bathtub shaped hazard rates among other things. Here, we consider estimation of the PDF and the CDF of the Weibull extension model. The following estimators are considered: uniformly minimum variance unbiased (UMVU) estimator, maximum likelihood (ML) estimator, percentile (PC) estimator, least squares (LS) estimator, and weighted least squares (WLS) estimator. Analytical expressions are derived for the bias and the mean squared error. Simulation studies and real data applications show that the ML estimator performs better than others.     

Rank estimation of regression coefficients using iterated reweighted least squares

This paper is concerned with the rank estimator for the parameter vector β in a linear model which is obtained by the minimization of a rank dispersion function. The rank estimator has many advantages over the regular least squares estimator, but the inaccessibility of software to carry out its computation has limited its use. An iterated reweighted least squares algorithm is presented for the computation of the rank estimator. The method is simple in concept and can be carried out readily with a wide variety of statistical software. Details of the method are discussed along with some results on its asymptotic distribution and numerical stability. Some examples are presented to show advantages of the rank method.     

空间回归模型估计中的最小二乘法

空间回归模型由于引入了空间地理信息而使得其参数估计变得复杂,因为主要采用最大似然法,致使一般人认为在空间回归模型参数估计中不存在最小二乘法。通过分析空间回归模型的参数估计技术,研究发现,最小二乘法和最大似然法分别用于估计空间回归模型的不同的参数,只有将两者结合起来才能快速有效地完成全部的参数估计。数理论证结果表明,空间回归模型参数最小二乘估计量是最佳线性无偏估计量。空间回归模型的回归参数可以在估计量为正态性的条件下而实施显著性检验,而空间效应参数则不可以用此方法进行检验。     

Consistency of the structured total least squares estimator in a multivariate errors-in-variables model

The structured total least squares estimator, defined via a constrained optimization problem, is a generalization of the total least squares estimator when the data matrix and the applied correction satisfy given structural constraints. In the paper, an affine structure with additional assumptions is considered. In particular, Toeplitz and Hankel structured, noise free and unstructured blocks are allowed simultaneously in the augmented data matrix. An equivalent optimization problem is derived that has as decision variables only the estimated parameters. The cost function of the equivalent problem is used to prove consistency of the structured total least squares estimator. The results for the general affine structured multivariate model are illustrated by examples of special models. Modification of the results for block-Hankel/Toeplitz structures is also given. As a by-product of the analysis of the cost function, an iterative algorithm for the computation of the structured total least squares estimator is proposed.     

Penalized weighted composite quantile regression in the linear regression model with heavy-tailed autocorrelated errors

In this paper, a penalized weighted composite quantile regression estimation procedure is proposed to estimate unknown regression parameters and autoregression coefficients in the linear regression model with heavy-tailed autoregressive errors. Under some conditions, we show that the proposed estimator possesses the oracle properties. In addition, we introduce an iterative algorithm to achieve the proposed optimization problem, and use a data-driven method to choose the tuning parameters. Simulation studies demonstrate that the proposed new estimation method is robust and works much better than the least squares based method when there are outliers in the dataset or the autoregressive error distribution follows heavy-tailed distributions. Moreover, the proposed estimator works comparably to the least squares based estimator when there are no outliers and the error is normal. Finally, we apply the proposed methodology to analyze the electricity demand dataset.     

Second-order least squares estimation of censored regression models

This paper proposes the second-order least squares estimation, which is an extension of the ordinary least squares method, for censored regression models where the error term has a general parametric distribution (not necessarily normal). The strong consistency and asymptotic normality of the estimator are derived under fairly general regularity conditions. We also propose a computationally simpler estimator which is consistent and asymptotically normal under the same regularity conditions. Finite sample behavior of the proposed estimators under both correctly and misspecified models are investigated through Monte Carlo simulations. The simulation results show that the proposed estimator using optimal weighting matrix performs very similar to the maximum likelihood estimator, and the estimator with the identity weight is more robust against the misspecification.     

Estimation and Simulation of the Riesz-Bessel Distribution

ABSTRACT

In this article, further properties of the Riesz-Bessel distribution are provided. These properties allow for the simulation of random variables from the Riesz-Bessel distribution. Estimation is addressed by nonlinear generalized least squares regression on the empirical characteristic function. The estimator is seen to approximate the maximum likelihood estimator. The distribution is illustrated with financial data.     

Multivariate regression models for discrete and continuous repeated measurements

A general class of multivariate regression models is considered for repeated measurements with discrete and continuous outcome variables. The proposed model is based on the seemingly unrelated regression model (Zellner, 1962) and an extension of the model of Park and Woolson(1992). The regression parameters of the model are consistently estimated using the two-stage least squares method. When the out come variables are multivariate normal, the two-stage estimator reduces to Zellner’s two-stage estimator. As a special case, we consider the marginal distribution described by Liang and Zeger (1986). Under this this distributional assumption, we show that the two-stage estimator has similar asymptotic properties and comparable small sample properties to Liang and Zeger's estimator. Since the proposed approach is based on the least squares method, however, any distributional assumption is not required for variables outcome variables. As a result, the proposed estimator is more robust to the marginal distribution of outcomes.     

Measuring the degree of severity of heteroskedasticity and the choice between the ols estimator and the 2sae

This paper dwells on the choice between the ordinary least squares and the estimated generalized least squares estimators when the presence of heteroskedasticity is suspected. Since the estimated generalized least squares estimator does not dominate the ordinary least squares estimator completely over the whole parameter space, it is of interest to the researcher to know in advance whether the degree of severity of heteroskedasticity is such that OLS estimator outperforms the estimated generalized least squares (or 2SAE). Casting the problem in the non-spherical error mold and exploiting the principle underlying the Bayesian pretest estimator, an intuitive non-mathematical procedure is proposed to serve as an aid to the researcher in deciding when to use either the ordinary least squares (OLS) or the estimated generalized least squares (2SAE) estimators.     

Principal components regression estimator and a test for the restrictions

In this article, we introduce restricted principal components regression (RPCR) estimator by combining the approaches followed in obtaining the restricted least squares estimator and the principal components regression estimator. The performance of the RPCR estimator with respect to the matrix and the generalized mean square error are examined. We also suggest a testing procedure for linear restrictions in principal components regression by using singly and doubly non-central F distribution.     

Minimax Linear Smoothers

We consider the problem of estimating the mean of a multivariate distribution. As a general alternative to penalized least squares estimators, we consider minimax estimators for squared error over a restricted parameter space where the restriction is determined by the penalization term. For a quadratic penalty term, the minimax estimator among linear estimators can be found explicitly. It is shown that all symmetric linear smoothers with eigenvalues in the unit interval can be characterized as minimax linear estimators over a certain parameter space where the bias is bounded. The minimax linear estimator depends on smoothing parameters that must be estimated in practice. Using results in Kneip (1994), this can be done using Mallows' C _L -statistic and the resulting adaptive estimator is now asymptotically minimax linear. The minimax estimator is compared to the penalized least squares estimator both in finite samples and asymptotically.