Maximum likelihood revisited under a semi-parametric context - estimation of the tail index

In this paper, and in a context of regularly varying tails, we study computationally the classical Maximum Likelihood (ML) estimator based on the Paretian behaviour of the excesses over a high threshold, denoted PML-estimator, a type II Censoring estimator based specifically on a Fréchet parent, denoted CENS-estimator, and two ML estimators based on the scaled log-spacings, and denoted SLS-estimators. These estimators are considered under a semi-parametric set-up, and compared with the classical Hill estimator and a Generalized Jackknife (GJ) estimator, which has essentially in mind a reduction of the bias of Hill's estimator.     

A note on properties of spatial yule-walker estimators

For two-dimensional spatial autoregressive (AR) models, asymptotic properties of the spatial Yule-Walker (YW) estimators (Tjøstheim, 1978) are studied. These estimators although consistent, are shown to be asymptotically biased. Estimators from the first-order spatial bilateral AR model are looked at in more detail and the spatial YW estimators for this model are compared with the exact maximum likelihood estimators. Small sample properties of both estimators are also discussed briefly and some simulation results are presented.     

Nonparametric maximum likelihood estimators for ar and ma time series

The problem of estimating the parameters of moving average or autoregressive time series is studied when the error distribution is completely unknown. Four nonparametric maximum likelihood estimators (NPMLE) are presented for this purpose. These estimators are compared with the classical moment and least squares estimators in a simulation study. The behavior of these NPMLEs is much better than the classical ones, suggesting that they should be used extensively when no parametric information is known in advance about the error distribution. An application of these estimators to coal mining accidents data is also included.     

Editorial collaborators

A simulation study of the binomial-logit model with correlated random effects is carried out based on the generalized linear mixed model (GLMM) methodology. Simulated data with various numbers of regression parameters and different values of the variance component are considered. The performance of approximate maximum likelihood (ML) and residual maximum likelihood (REML) estimators is evaluated. For a range of true parameter values, we report the average biases of estimators, the standard error of the average bias and the standard error of estimates over the simulations. In general, in terms of bias, the two methods do not show significant differences in estimating regression parameters. The REML estimation method is slightly better in reducing the bias of variance component estimates.     

To be or not to be valid in testing the significance of the slope in simple quantitative linear models with autocorrelated errors

In this article, the validity of procedures for testing the significance of the slope in quantitative linear models with one explanatory variable and first-order autoregressive [AR(1)] errors is analyzed in a Monte Carlo study conducted in the time domain. Two cases are considered for the regressor: fixed and trended versus random and AR(1). In addition to the classical t -test using the Ordinary Least Squares (OLS) estimator of the slope and its standard error, we consider seven t -tests with n-2\,\hbox{df} built on the Generalized Least Squares (GLS) estimator or an estimated GLS estimator, three variants of the classical t -test with different variances of the OLS estimator, two asymptotic tests built on the Maximum Likelihood (ML) estimator, the F -test for fixed effects based on the Restricted Maximum Likelihood (REML) estimator in the mixed-model approach, two t -tests with n - 2 df based on first differences (FD) and first-difference ratios (FDR), and four modified t -tests using various corrections of the number of degrees of freedom. The FDR t -test, the REML F -test and the modified t -test using Dutilleul's effective sample size are the most valid among the testing procedures that do not assume the complete knowledge of the covariance matrix of the errors. However, modified t -tests are not applicable and the FDR t -test suffers from a lack of power when the regressor is fixed and trended ( i.e. , FDR is the same as FD in this case when observations are equally spaced), whereas the REML algorithm fails to converge at small sample sizes. The classical t -test is valid when the regressor is fixed and trended and autocorrelation among errors is predominantly negative, and when the regressor is random and AR(1), like the errors, and autocorrelation is moderately negative or positive. We discuss the results graphically, in terms of the circularity condition defined in repeated measures ANOVA and of the effective sample size used in correlation analysis with autocorrelated sample data. An example with environmental data is presented.     

On the Estimation of Parameters of Kumaraswamy-G Distributions

Cordeiro and de Castro proposed a new family of generalized distributions based on the Kumaraswamy distribution (denoted as Kw-G). Nadarajah et al. showed that the density function of the new family of distributions can be expressed as a linear combination of the density of exponentiated family of distributions. They derived some properties of Kw-G distributions and discussed estimation of parameters using the maximum likelihood (ML) method. Cheng and Amin and Ranneby introduced a new method of estimating parameters based on Kullback–Leibler divergence (the maximum spacing (MSP) method). In this article, the estimates of parameters of Kw-G distributions are obtained using the MSP method. For some special Kw-G distributions, the new estimators are compared with ML estimators. It is shown by simulations and a real data application that MSP estimators have better properties than ML estimators.     

Asymptotics for REML estimation of spatial covariance parameters

In agricultural field trials, restricted maximum likelihood estimation (REML) of the spatial covariance parameters is often preferred to maximum likelihood. Although it has either been conjectured or assumed that REML estimators are asymptotically Gaussian, conditions under which such asymptotic results hold are clearly needed. This article gives checkable conditions for spatial regression when sampling locations are either on a rectangular grid or are irregularly spaced but satisfy certain growth conditions.     

On the robustness properties for maximum likelihood estimators of parameters in exponential power and generalized T distributions*

Abstract

Examining the robustness properties of maximum likelihood (ML) estimators of parameters in exponential power and generalized t distributions has been considered together. The well-known asymptotic properties of ML estimators of location, scale and added skewness parameters in these distributions are studied. The ML estimators for location, scale and scale variant (skewness) parameters are represented as an iterative reweighting algorithm (IRA) to compute the estimates of these parameters simultaneously. The artificial data are generated to examine performance of IRA for ML estimators of parameters simultaneously. We make a comparison between these two distributions to test the fitting performance on real data sets. The goodness of fit test and information criteria approve that robustness and fitting performance should be considered together as a key for modeling issue to have the best information from real data sets.     

On the difference between ML and REML estimators in the modelling of multivariate longitudinal data

A random effects model is examined in the multivariate setting where more than one characteristics are measured at each time point. ML and REML estimators are obtained under the restriction that estimates of variance matrices being at least p.s.d. It is shown that REML has greater probability of giving full rank estimates of variance components matrices but as regards the efficiency in the estimation of the location parameter, correct specification of the number of random effects is needed. In general, REML provides larger estimates of variance of model parameters than ML.     

Statistical inference for α-series process with the inverse Gaussian distribution

Statistical inferences for the geometric process (GP) are derived when the distribution of the first occurrence time is assumed to be inverse Gaussian (IG). An α-series process, as a possible alternative to the GP, is introduced since the GP is sometimes inappropriate to apply some reliability and scheduling problems. In this study, statistical inference problem for the α-series process is considered where the distribution of first occurrence time is IG. The estimators of the parameters α, μ, and σ² are obtained by using the maximum likelihood (ML) method. Asymptotic distributions and consistency properties of the ML estimators are derived. In order to compare the efficiencies of the ML estimators with the widely used nonparametric modified moment (MM) estimators, Monte Carlo simulations are performed. The results showed that the ML estimators are more efficient than the MM estimators. Moreover, two real life datasets are given for application purposes.     

Stepwise Regression in Mixed Quantitative Linear Models with Autocorrelated Errors

ABSTRACT

In the stepwise procedure of selection of a fixed or a random explanatory variable in a mixed quantitative linear model with errors following a Gaussian stationary autocorrelated process, we have studied the efficiency of five estimators relative to Generalized Least Squares (GLS): Ordinary Least Squares (OLS), Maximum Likelihood (ML), Restricted Maximum Likelihood (REML), First Differences (FD), and First-Difference Ratios (FDR). We have also studied the validity and power of seven derived testing procedures, to assess the significance of the slope of the candidate explanatory variable x ₂ to enter the model in which there is already one regressor x ₁. In addition to five testing procedures of the literature, we considered the FDR t-test with n ? 3 df and the modified t-test with n? ? 3 df for partial correlations, where n? is Dutilleul's effective sample size. Efficiency, validity, and power were analyzed by Monte Carlo simulations, as functions of the nature, fixed vs. random (purely random or autocorrelated), of x ₁ and x ₂, the sample size and the autocorrelation of random terms in the regression model. We report extensive results for the autocorrelation structure of first-order autoregressive [AR(1)] type, and discuss results we obtained for other autocorrelation structures, such as spherical semivariogram, first-order moving average [MA(1)] and ARMA(1,1), but we could not present because of space constraints. Overall, we found that:

1. the efficiency of slope estimators and the validity of testing procedures depend primarily on the nature of x ₂, but not on that of x ₁;

2. FDR is the most inefficient slope estimator, regardless of the nature of x ₁ and x ₂;

3. REML is the most efficient of the slope estimators compared relative to GLS, provided the specified autocorrelation structure is correct and the sample size is large enough to ensure the convergence of its optimization algorithm;

4. the FDR t-test, the modified t-test and the REML t-test are the most valid of the testing procedures compared, despite the inefficiency of the FDR and OLS slope estimators for the former two;

5. the FDR t-test, however, suffers from a lack of power that varies with the nature of x ₁ and x ₂; and

6. the modified t-test for partial correlations, which does not require the specification of an autocorrelation structure, can be recommended when x ₁ is fixed or random and x ₂ is random, whether purely random or autocorrelated. Our results are illustrated by the environmental data that motivated our work.

     

Estimation of the mean of the exponential distribution using moving extremes ranked set sampling

Moving Extremes Ranked Set Sampling (MERSS) is a useful modification of Ranked Set Sampling (RSS). Unlike RSS, MERSS allows for an increase of set size without introducing too much ranking error. The method is considered parametrically under exponential distribution. Maximum likelihood estimator (MLE), and a modified MLE are considered and their properties are studied. The method is studied under both perfect and imperfect ranking (with error in ranking). It appears that these estimators can be real competitors to the MLE using the usual simple random sampling (SRS).     

A consistency result in general censoring models

In this paper we prove a consistency result for sieved maximum likelihood estimators of the density in general random censoring models with covariates. The proof is based on the method of functional estimation. The estimation error is decomposed in a deterministic approximation error and the stochastic estimation error. The main part of the proof is to establish a uniform law of large numbers for the conditional log-likelihood functional, by using results and techniques from empirical process theory.     

Negative moments of a modified power series distribution and bias of the maximum likelihood estimator

The class of Modified Power Series distributions (MPSD) containing Lagrangian Poisson (LPD) (Consul and Jain, 1973) and Lagrangian binomial distributions (LBD) (Jain and Consul, 1971) was studied by Gupta (1974). We investigate the problem of finding the negative momentsE[X^-r ], of displaced and decapitated Modified Power Series Distributions. We derive the relationship between rand (r-1) negative moments. The negative moments of the decapitated and displaced LPD are obtained. These results are, then, used to find the exact amount of bias in the ML estimators of the parameters in the LPD and the LBD. We have also given the variances of the ML estimator and the minimum variance unbiased estimator of the parameter in the LPD.     

ESTIMATION OF PARAMETERS IN THE MARGINAL FISHER DISTRIBUTION1

The Fisher distribution is frequently used as a model for the probability distribution of directional data, which may be specified either in terms of unit vectors or angular co-ordinates (co-latitude and azimuth). If, in practical situations, only the co-latitudes can be observed, the available data must be regarded as a sample from the corresponding marginal distribution. This paper discusses the estimation by Maximum Likelihood (ML) and the Method of Moments of the two parameters of this marginal Fisher distribution. The moment estimators are generally simpler to compute than the ML estimators, and have high asymptotic efficiency.     

Efficient mean estimation in log-normal linear models

Log-normal linear models are widely used in applications, and many times it is of interest to predict the response variable or to estimate the mean of the response variable at the original scale for a new set of covariate values. In this paper we consider the problem of efficient estimation of the conditional mean of the response variable at the original scale for log-normal linear models. Several existing estimators are reviewed first, including the maximum likelihood (ML) estimator, the restricted ML (REML) estimator, the uniformly minimum variance unbiased (UMVU) estimator, and a bias-corrected REML estimator. We then propose two estimators that minimize the asymptotic mean squared error and the asymptotic bias, respectively. A parametric bootstrap procedure is also described to obtain confidence intervals for the proposed estimators. Both the new estimators and the bootstrap procedure are very easy to implement. Comparisons of the estimators using simulation studies suggest that our estimators perform better than the existing ones, and the bootstrap procedure yields confidence intervals with good coverage properties. A real application of estimating the mean sediment discharge is used to illustrate the methodology.     

Maximum likelihood estimation of variance components of heteroscedastic random anova model

the estimation of variance components of heteroscedastic random model is discussed in this paper. Maximum Likelihood (ML) is described for one-way heteroscedastic random models. The proportionality condition that cell variance is proportional to the cell sample size, is used to eliminate the efffect of heteroscedasticity. The algebraic expressions of the estimators are obtained for the model. It is seen that the algebraic expressions of the estimators depend mainly on the inverse of the variance-covariance matrix of the observation vector. So, the variance-covariance matrix is obtained and the formulae for the inversions are given. A Monte Carlo study is conducted. Five different variance patterns with different numbers of cells are considered in this study. For each variance pattern, 1000 Monte Carlo samples are drawn. Then the Monte Carlo biases and Monte Carlo MSE’s of the estimators of variance components are calculated. In respect of both bias and MSE, the Maximum Likelihood (ML) estimators of variance components are found to be sufficiently good.     

Estimation and test of several multivariate normal means under an order restriction when the dimension is larger than two

Suppose that an order restriction is imposed among several p-variate normal mean vectors. We are interested in the problems of estimating these mean vectors and testing their homogeneity under this restriction. These problems are multivariate extensions of Bartholomew's (1959) ones. For the bivariate case, these problems have been studied by Sasabuchi et al. (1983) and (1998) and some others. In the present paper we examine the convergence of an iterative algorithm for computing the maximum likelihood estimator when p is larger than two. We also study some test procedures for testing homogeneity when p is larger than two.     

An em algorithm for a semiparametric finite mixture model

We propose a semiparametric version of the EM algorithm under the semiparametric mixture model introduced by Anderson (1979, Biometrika , 66 , 17-26). It is shown that the sequence of proposed EM iterates, irrespective of the starting value, converges to the maximum semiparametric likelihood estimator of the vector of parameters in the semiparametric mixture model. The proposed EM algorithm preserves the appealing monotone convergence property of the standard EM algorithm and can be implemented by employing the standard logistic regression program. We present one example to demonstrate the performance of the proposed EM algorithm.     

Multiple Regression Model Averaging and the Focused Information Criterion With an Application to Portfolio Choice

ABSTRACT

We consider multiple regression (MR) model averaging using the focused information criterion (FIC). Our approach is motivated by the problem of implementing a mean-variance portfolio choice rule. The usual approach is to estimate parameters ignoring the intention to use them in portfolio choice. We develop an estimation method that focuses on the trading rule of interest. Asymptotic distributions of submodel estimators in the MR case are derived using a localization framework. The localization is of both regression coefficients and error covariances. Distributions of submodel estimators are used for model selection with the FIC. This allows comparison of submodels using the risk of portfolio rule estimators. FIC model averaging estimators are then characterized. This extension further improves risk properties. We show in simulations that applying these methods in the portfolio choice case results in improved estimates compared with several competitors. An application to futures data shows superior performance as well.