Maximum likelihood estimation for arma models in the presence of ARMA errors

An ARMA(p, q) process observed with an ARMA(c, d) error has an ARMA (p + c, k) representation with k = max(c + q, p + d) whose parameters satisfy some nonlinear constraints. Identification of the model is discussed. We develop Newton-Raphson estimators for the ARMA(p + c, k) process which take advantage of the information contained in the nonlinear restrictions. Explicit expressions for the derivatives of the restrictions are derived.     

Shrinkage and penalized estimators in weighted least absolute deviations regression models

In this paper, we consider the estimation problem of the weighted least absolute deviation (WLAD) regression parameter vector when there are some outliers or heavy-tailed errors in the response and the leverage points in the predictors. We propose the pretest and James–Stein shrinkage WLAD estimators when some of the parameters may be subject to certain restrictions. We derive the asymptotic risk of the pretest and shrinkage WLAD estimators and show that if the shrinkage dimension exceeds two, the asymptotic risk of the shrinkage WLAD estimator is strictly less than the unrestricted WLAD estimator. On the other hand, the risk of the pretest WLAD estimator depends on the validity of the restrictions on the parameters. Furthermore, we study the WLAD absolute shrinkage and selection operator (WLAD-LASSO) and compare its relative performance with the pretest and shrinkage WLAD estimators. A simulation study is conducted to evaluate the performance of the proposed estimators relative to that of the unrestricted WLAD estimator. A real-life data example using body fat study is used to illustrate the performance of the suggested estimators.     

Generalized preliminary test stochastic restricted estimator in the linear regression model

ABSTRACT

In this paper, we propose three generalized estimators, namely, generalized unrestricted estimator (GURE), generalized stochastic restricted estimator (GSRE), and generalized preliminary test stochastic restricted estimator (GPTSRE). The GURE can be used to represent the ridge estimator, almost unbiased ridge estimator (AURE), Liu estimator, and almost unbiased Liu estimator. When stochastic restrictions are available in addition to the sample information, the GSRE can be used to represent stochastic mixed ridge estimator, stochastic restricted Liu estimator, stochastic restricted almost unbiased ridge estimator, and stochastic restricted almost unbiased Liu estimator. The GPTSRE can be used to represent the preliminary test estimators based on mixed estimator. Using the GPTSRE, the properties of three other preliminary test estimators, namely preliminary test stochastic mixed ridge estimator, preliminary test stochastic restricted almost unbiased Liu estimator, and preliminary test stochastic restricted almost unbiased ridge estimator can also be discussed. The mean square error matrix criterion is used to obtain the superiority conditions to compare the estimators based on GPTSRE with some biased estimators for the two cases for which the stochastic restrictions are correct, and are not correct. Finally, a numerical example and a Monte Carlo simulation study are done to illustrate the theoretical findings of the proposed estimators.     

Shrinkage and pretest estimators for longitudinal data analysis under partially linear models

In this paper, we develop marginal analysis methods for longitudinal data under partially linear models. We employ the pretest and shrinkage estimation procedures to estimate the mean response parameters as well as the association parameters, which may be subject to certain restrictions. We provide the analytic expressions for the asymptotic biases and risks of the proposed estimators, and investigate their relative performance to the unrestricted semiparametric least-squares estimator (USLSE). We show that if the dimension of association parameters exceeds two, the risk of the shrinkage estimators is strictly less than that of the USLSE in most of the parameter space. On the other hand, the risk of the pretest estimator depends on the validity of the restrictions of association parameters. A simulation study is conducted to evaluate the performance of the proposed estimators relative to that of the USLSE. A real data example is applied to illustrate the practical usefulness of the proposed estimation procedures.     

Shrinkage Estimation of the Memory Parameter in Stationary Gaussian Processes

The correct and efficient estimation of memory parameters in a stationary Gaussian processes is an important issue, since otherwise, forecasts based on the resulting time series would be misleading. On the other hand, if the memory parameters are suspected to fall in a smaller subspace through some hypothesis restrictions, it becomes a hard decision whether to use estimators based on the restricted spaces or to use unrestricted estimators over the full parameter space. In this article, we propose James-Stein-type estimators of the memory parameters of a stationary Gaussian times series process, which can efficiently incorporate the hypothetical restrictions. We show theoretically that the proposed estimators are more efficient than the usual unrestricted maximum likelihood estimators over the entire parameter space.     

On the estimation of reliabilty function in a Weibull lifetime distribution

The estimation of the reliability function of the Weibull lifetime model is considered in the presence of uncertain prior information (not in the form of prior distribution) on the parameter of interest. This information is assumed to be available in some sort of a realistic conjecture. In this article, we focus on how to combine sample and non-sample information together in order to achieve improved estimation performance. Three classes of point estimatiors, namely, the unrestricted estimator, the shrinkage estimator and shrinkage preliminary test estimator (SPTE) are proposed. Their asymptotic biases and mean-squared errors are derived and compared. The relative dominance picture of the estimators is presented. Interestingly, the proposed SPTE dominates the unrestricted estimator in a range that is wider than that of the usual preliminary test estimator. A small-scale simulation experiment is used to examine the small sample properties of the proposed estimators. Our simulation investigations have provided strong evidence that corroborates with asymptotic theory. The suggested estimation methods are applied to a published data set to illustrate the performance of the estimators in a real-life situation.     

Maximum likelihood estimators for ARMA and ARFIMA models: a Monte Carlo study

We analyze by simulation the properties of two time domain and two frequency domain estimators for low-order autoregressive fractionally integrated moving-average Gaussian models, ARFIMA (p,d,q). The estimators considered are the exact maximum likelihood for demeaned data (EML) the associated modified profile likelihood (MPL) and the Whittle estimator with (WLT) and without tapered data (WL). Length of the series is 100. The estimators are compared in terms of pile-up effect, mean square error, bias, and empirical confidence level. The tapered version of the Whittle likelihood turns out to be a reliable estimator for ARMA and ARFIMA models. Its small losses in performance in case of ‘well-behaved’ models are compensated sufficiently in more ‘difficult’ models. The modified profile likelihood is an alternative to the WLT but is computationally more demanding. It is either equivalent to the EML or more favorable than the EML. For fractionally integrated models, particularly, it dominates clearly the EML. The WL has serious deficiencies for large ranges of parameters, and so cannot be recommended in general. The EML, on the other hand, should only be used with care for fractionally integrated models due to its potential large negative bias of the fractional integration parameter. In general, one should proceed with caution for ARMA(1,1) models with almost canceling roots, and, in particular, in case of the EML and the MPL for inference in the vicinity of a moving-average root of +1.     

Pooling means under uncertain prior information with application to discrete distributions

The problem of pooling means is considered based on two samples in presence of the uncertain prior information that these samples are taken from possibly identical populations. Two discrete models, Poisson and binomial are considered in particular. Three estimators, i.e. the unrestricted estimator, shrinkage restricted estimator and estimators based on preliminary test are proposed. Their asymptotic mean squared errors are derived and compared. It is demonstrated via asymptotic results that the range of the parameter space in which shrinkage preliminary test estimator dominates the unrestricted estimator is wider than that of the usual preliminary test estimator. A Monte Carlo study for Poisson model is presented to compare the performance of the estimators for small samples.     

Improved pretest nonparametric estimation in a multivariate regression model

Shrinkage pretest nonparametric estimation of the location parameter vector in a multivariate regression model is considered when nonsample information (NSI) about the regression parameters is available. By using the quadratic risk criterion, the dominance of the pretest estimators over the usual estimators has been investigated. We demonstrate analytically and computationally that the proposed improved pretest estimator establishes a wider dominance range for the parameter under consideration than that of the usual pretest estimator in which it is superior over the unrestricted estimator.     

Estimation in long memory time series models

A regression type estimator of the parameter d in fractionally differenced ARMA (p,q) processes is presented. The proposed estimator is shown to be mean square consistent. Its performance is compared with some of the existing estimators via a simulation study.     

The relative efficiency of the restricted estimators in linear regression models

This paper deals with the problem of multicollinearity in a multiple linear regression model with linear equality restrictions. The restricted two parameter estimator which was proposed in case of multicollinearity satisfies the restrictions. The performance of the restricted two parameter estimator over the restricted least squares (RLS) estimator and the ordinary least squares (OLS) estimator is examined under the mean square error (MSE) matrix criterion when the restrictions are correct and not correct. The necessary and sufficient conditions for the restricted ridge regression, restricted Liu and restricted shrunken estimators, which are the special cases of the restricted two parameter estimator, to have a smaller MSE matrix than the RLS and the OLS estimators are derived when the restrictions hold true and do not hold true. Theoretical results are illustrated with numerical examples based on Webster, Gunst and Mason data and Gorman and Toman data. We conduct a final demonstration of the performance of the estimators by running a Monte Carlo simulation which shows that when the variance of the error term and the correlation between the explanatory variables are large, the restricted two parameter estimator performs better than the RLS estimator and the OLS estimator under the configurations examined.     

Improved two-parameter estimators for the negative binomial and Poisson regression models

Negative binomial regression (NBR) and Poisson regression (PR) applications have become very popular in the analysis of count data in recent years. However, if there is a high degree of relationship between the independent variables, the problem of multicollinearity arises in these models. We introduce new two-parameter estimators (TPEs) for the NBR and the PR models by unifying the two-parameter estimator (TPE) of Özkale and Kaç?ranlar [The restricted and unrestricted two-parameter estimators. Commun Stat Theory Methods. 2007;36:2707–2725]. These new estimators are general estimators which include maximum likelihood (ML) estimator, ridge estimator (RE), Liu estimator (LE) and contraction estimator (CE) as special cases. Furthermore, biasing parameters of these estimators are given and a Monte Carlo simulation is done to evaluate the performance of these estimators using mean square error (MSE) criterion. The benefits of the new TPEs are also illustrated in an empirical application. The results show that the new proposed TPEs for the NBR and the PR models are better than the ML estimator, the RE and the LE.     

Combining poisson means

The problem of estimating the Poisson mean is considered based on the two samples in the presence of uncertain prior information (not in the form of distribution) that two independent random samples taken from two possibly identical Poisson populations. The parameter of interest is λ₁ from population I. Three estimators, i.e. the unrestricted estimator, restricted estimator and preliminary test estimator are proposed. Their asymptotic mean squared errors are derived and compared; parameter regions have been found for which restricted and preliminary test estimators are always asymptotically more efficient than the classical estimator. The relative dominance picture of the estimators is presented. Maximum and minimum asymptotic efficiencies of the estimators relative to the classical estimator are tabulated. A max-min rule for the size of the preliminary test is also discussed. A Monte Carlo study is presented to compare the performance of the estimator with that of Kale and Bancroft (1967).     

Super efficient frequency estimation

In this paper we propose a modified Newton-Raphson method to obtain super efficient estimators of the frequencies of a sinusoidal signal in presence of stationary noise. It is observed that if we start from an initial estimator with convergence rate O_p(n⁻¹) and use Newton-Raphson algorithm with proper step factor modification, then it produces super efficient frequency estimator in the sense that its asymptotic variance is lower than the asymptotic variance of the corresponding least squares estimator. The proposed frequency estimator is consistent and it has the same rate of convergence, namely O_p(n^−3/2), as the least squares estimator. Monte Carlo simulations are performed to observe the performance of the proposed estimator for different sample sizes and for different models. The results are quite satisfactory. One real data set has been analyzed for illustrative purpose.     

The Almon two parameter estimator for the distributed lag models

The two parameter estimator proposed by Özkale and Kaç?ranlar [The restricted and unrestricted two parameter estimators. Comm Statist Theory Methods. 2007;36(15):2707–2725] is a general estimator which includes the ordinary least squares, the ridge and the Liu estimators as special cases. In the present paper we introduce Almon two parameter estimator based on the two parameter estimation procedure to deal with the problem of multicollinearity for the distiributed lag models. This estimator outperforms the Almon estimator according to the matrix mean square error criterion. Moreover, a numerical example and a Monte Carlo simulation experiment are presented by using different estimators of the biasing parameters.     

Admissibility of linear estimators with respect to restricted parameter sets

In this paper the admissibility of a linear estimator for a linear regression parameter is characterized for such cases, where the considered parameter varies in an ellipsoid. We obtain a certain subset of the set of all linear estimators which are admissible with respect to the unrestricted parameter set. Furthermore, various linear estimators which have been proposed for improving the least squares estimator in cases of a restricted parameter set are investigated for admissibility. It turns out that only some shrunken estimators and some estimators of ridge type are admissible, whereas the KUKS-OLMAN estimator and all estimators of MARQUARDT type can be improved.     

Estimation of two order restricted normal means with unknown and possibly unequal variances

Estimation of each of and linear functions of two order restricted normal means is considered when variances are unknown and possibly unequal. We replace unknown variances with sample variances and construct isotonic regression estimators, which we call in our paper the plug-in estimators, to estimate ordered normal means. Under squared error loss, a necessary and sufficient condition is given for the plug-in estimators to improve upon the unrestricted maximum likelihood estimators uniformly. As for the estimation of linear functions of ordered normal means, we also show that when variances are known, the restricted maximum likelihood estimator always improves upon the unrestricted maximum likelihood estimator uniformly, but when variances are unknown, the plug-in estimator does not always improve upon the unrestricted maximum likelihood estimator uniformly.     

Preliminary estimators for robust non-linear regression estimation

In this paper, the robustness of weighted non-linear least-squares estimation based on some preliminary estimators is examined. The preliminary estimators are the Lnorm estimates proposed by Schlossmacher, by El-Attar et al., by Koenker and Park, and by Lawrence and Arthur. A numerical example is presented to compare the robustness of the weighted non-linear least-squares approach when based on the preliminary estimators of Schlossmacher (HS), El-Attar et al. (HEA), Koenker and Park (HKP), and Lawrence and Arthur (HLA). The study shows that the HEA estimator is as robust as the HKP estimator. However, the HEA estimator posed certain computational problems and required more storage and computing time.     

Merging data from multiple sources: pretest and shrinkage perspectives

In this article, we have developed asymptotic theory for the simultaneous estimation of the k means of arbitrary populations under the common mean hypothesis and further assuming that corresponding population variances are unknown and unequal. The unrestricted estimator, the Graybill-Deal-type restricted estimator, the preliminary test, and the Stein-type shrinkage estimators are suggested. A large sample test statistic is also proposed as a pretest for testing the common mean hypothesis. Under the sequence of local alternatives and squared error loss, we have compared the asymptotic properties of the estimators by means of asymptotic distributional quadratic bias and risk. Comprehensive Monte-Carlo simulation experiments were conducted to study the relative risk performance of the estimators with reference to the unrestricted estimator in finite samples. Two real-data examples are also furnished to illustrate the application of the suggested estimation strategies.     

On the sensitivity of pre-test estimators to covariance misspecification

In statistical and econometric practice it is not uncommon to find that regression parameter estimates obtained using estimated generalized least squares (EGLS) do not differ much from those obtained through ordinary least squares (OLS), even when the assumption of spherical errors is violated. To investigate if one could ignore non-spherical errors, and legitimately continue with OLS estimation under the non-spherical disturbance setting, Banerjee and Magnus (1999) developed statistics to measure the sensitivity of the OLS estimator to covariance misspecification. Wan et al. (2007) generalized this work by allowing for linear restrictions on the regression parameters. This paper extends the aforementioned studies by exploring the sensitivity of the equality restrictions pre-test estimator to covariance misspecification. We find that the pre-test estimators can be very sensitive to covariance misspecification, and the degree of sensitivity of the pre-test estimator often lies between that of its unrestricted and restricted components. In addition, robustness to non-normality is investigated. It is found that existing results remain valid if elliptically symmetric, instead of normal, errors are assumed.