Efficiency of weighted averages

When combining estimates of a common parameter (of dimension d?1$d?1$) from independent data sets—as in stratified analyses and meta analyses—a weighted average, with weights ‘proportional’ to inverse variance matrices, is shown to have a minimal variance matrix (a standard fact when d=1$d=1$)—minimal in the sense that all convex combinations of the coordinates of the combined estimate have minimal variances. Minimum variance for the estimation of a single coordinate of the parameter can therefore be achieved by joint estimation of all coordinates using matrix weights. Moreover, if each estimate is asymptotically efficient within its own data set, then this optimally weighted average, with consistently estimated weights, is shown to be asymptotically efficient in the combined data set and avoids the need to merge the data sets and estimate the parameter in question afresh. This is so whatever additional non-common nuisance parameters may be in the models for the various data sets. A special case of this appeared in Fisher [1925. Theory of statistical estimation. Proc. Cambridge Philos. Soc. 22, 700–725.]: Optimal weights are ‘proportional’ to information matrices, and he argued that sample information should be used as weights rather than expected information, to maintain second-order efficiency of maximum likelihood. A number of special cases have appeared in the literature; we review several of them and give additional special cases, including stratified regression analysis—proportional-hazards, logistic or linear—, combination of independent ROC curves, and meta analysis. A test for homogeneity of the parameter across the data sets is also given.     

LEVERAGE ADJUSTMENTS FOR DISPERSION MODELLING IN GENERALIZED NONLINEAR MODELS

For normal linear models, it is generally accepted that residual maximum likelihood estimation is appropriate when covariance components require estimation. This paper considers generalized linear models in which both the mean and the dispersion are allowed to depend on unknown parameters and on covariates. For these models there is no closed form equivalent to residual maximum likelihood except in very special cases. Using a modified profile likelihood for the dispersion parameters, an adjusted score vector and adjusted information matrix are found under an asymptotic development that holds as the leverages in the mean model become small. Subsequently, the expectation of the fitted deviances is obtained directly to show that the adjusted score vector is unbiased at least to O(1/n) . Exact results are obtained in the single‐sample case. The results reduce to residual maximum likelihood estimation in the normal linear case.     

Testing Hypotheses for Variance Components in Mixed Linear Models

In the paper the problem of testing hypotheses for variance components in mixed linear models is considered. It is assumed that covariance matrices commute after using the usual invariance procedure with respect to the group of translations. The test for vanishing of single variance component is based on the locally best quadratic unbiased estimator of this component and rejects hypothesis if the ratio of positive and negative part of this estimator is sufficiently large. The power of this test with powers of other four tests for two-way classification models corresponding to block design is compared.     

Accent on Teaching Materials

Estimation of covariance components in the multivariate random-effect model with nested covariance structure is discussed. There are two covariance matrices to be estimated, namely, the between-group and the within-group covariance matrices. These two covariance matrices are most often estimated by forming a multivariate analysis of variance and equating mean square matrices to their expectations. Such a procedure involves taking the difference between the between-group mean square and the within-group mean square matrices, and often produces an estimated between-group covariance matrix that is not nonnegative definite. We present estimators of the two covariance matrices that are always proper covariance matrices. The estimators are the restricted maximum likelihood estimators if the random effects are normally distributed. The estimation procedure is extended to more complicated models, including the twofold nested and the mixed-effect models. A numerical example is presented to illustrate the use of the estimation procedure.     

Estimating the probability of winning (losing) in a gambler's ruin problem with applications

In a classical gambler's ruin problem, the distribution of the number of games lost till ruin is considered, which we call the lost game distribution (LGD). Some applications of LGD in the theory of queues, in the theory of epidemic and in certain clustering and branching models are mentioned. The maximum likelihood estimation of LGD in the framework of modified power series distribution (MPSD), introduced by the author (1974), is studied. The variance and bias of the MLE are given and the actual mean of the MLE is obtained by discussing the negative moments of the MPSD in general. The minimum variance unbiased estimator of θ^k (k≥1) is obtained employing the technique developed by the author (1977) for the class of MPSD.     

Maximum likelihood estimation of ARFIMA models with a Box-Cox transformation

In this paper we study the interaction between the estimation of the fractional differencing parameter d of ARFIMA models and the common practice of instantaneous transformation of the observed time series. At this aim, we first discuss the effect of a nonlinear transformation of the data on the identification of the process and on the estimate of d. Thus, we propose a joint estimation of the Box-Cox parameter and d by means of a modified normalized version of the Whittle likelihood. Then, the variance and covariance matrix of the parameters estimates is obtained. Finally, a Monte Carlo study is performed in order to check the behaviour of the proposed estimators in finite samples.The paper is the result of a joint research of the two authors. As far as it concerns this version of the work, A. DElia wrote Sects. 2, 3, 4, while D. Piccolo wrote Sects. 1, 5, 6.     

Inference on Nadarajah–Haghighi distribution with constant stress partially accelerated life tests under progressive type-II censoring

This paper presents methods of estimation of the parameters and acceleration factor for Nadarajah–Haghighi distribution based on constant-stress partially accelerated life tests. Based on progressive Type-II censoring, Maximum likelihood and Bayes estimates of the model parameters and acceleration factor are established, respectively. In addition, approximate confidence interval are constructed via asymptotic variance and covariance matrix, and Bayesian credible intervals are obtained based on importance sampling procedure. For comparison purpose, alternative bootstrap confidence intervals for unknown parameters and acceleration factor are also presented. Finally, extensive simulation studies are conducted for investigating the performance of the our results, and two data sets are analyzed to show the applicabilities of the proposed methods.     

Bayesian analysis of joint mean and covariance models for longitudinal data

Efficient estimation of the regression coefficients in longitudinal data analysis requires a correct specification of the covariance structure. If misspecification occurs, it may lead to inefficient or biased estimators of parameters in the mean. One of the most commonly used methods for handling the covariance matrix is based on simultaneous modeling of the Cholesky decomposition. Therefore, in this paper, we reparameterize covariance structures in longitudinal data analysis through the modified Cholesky decomposition of itself. Based on this modified Cholesky decomposition, the within-subject covariance matrix is decomposed into a unit lower triangular matrix involving moving average coefficients and a diagonal matrix involving innovation variances, which are modeled as linear functions of covariates. Then, we propose a fully Bayesian inference for joint mean and covariance models based on this decomposition. A computational efficient Markov chain Monte Carlo method which combines the Gibbs sampler and Metropolis–Hastings algorithm is implemented to simultaneously obtain the Bayesian estimates of unknown parameters, as well as their standard deviation estimates. Finally, several simulation studies and a real example are presented to illustrate the proposed methodology.     

Modified profile likelihood estimation for the Weibull regression models in survival analysis

In this study, adjustment of profile likelihood function of parameter of interest in presence of many nuisance parameters is investigated for survival regression models. Our objective is to extend the Barndorff–Nielsen’s technique to Weibull regression models for estimation of shape parameter in presence of many nuisance and regression parameters. We conducted Monte-Carlo simulation studies and a real data analysis, all of which demonstrate and suggest that the modified profile likelihood estimators outperform the profile likelihood estimators in terms of three comparison criterion: mean squared errors, bias and standard errors.     

Multilevel maximum likelihood estimation with application to covariance matrices

The asymptotic variance of the maximum likelihood estimate is proved to decrease when the maximization is restricted to a subspace that contains the true parameter value. Maximum likelihood estimation allows a systematic fitting of covariance models to the sample, which is important in data assimilation. The hierarchical maximum likelihood approach is applied to the spectral diagonal covariance model with different parameterizations of eigenvalue decay, and to the sparse inverse covariance model with specified parameter values on different sets of nonzero entries. It is shown computationally that using smaller sets of parameters can decrease the sampling noise in high dimension substantially.     

Non–zero inflated modified power series distributions

In this paper, we study the class of inflated modified power series distributions (IMPSD) where inflation occurs at any of the support points. This class include among other the generalized Poisson, the generalized negative binomial, the generalized logarithmic series and the lost games distributions. We give expressions for the moments, factorial moments and central moments of the IMPSD. The maximum likelihood estimation of the parameters of the IMPSD and the variance – covariance matrix of the estimators is obtained. We derive these estimators and their information matrices for mentioned above particular members of IMPSD class. The second part of this paper deals with the distribution of sum of independent and identically distributed random variables taking values s, s+1. s + 2, …, s ≥ 0, with modified power series distributions inflated at the point s.     

Some properties of tests for parameters that can be arbitrarily close to being unidentified

Confidence intervals for parameters that can be arbitrarily close to being unidentified are unbounded with positive probability [e.g. Dufour, J.-M., 1997. Some impossibility theorems in econometrics with applications to instrumental variables and dynamic models. Econometrica 65, 1365–1388; Pfanzagl, J. 1998. The nonexistence of confidence sets for discontinuous functionals. Journal of Statistical Planning and Inference 75, 9–20], and the asymptotic risks of their estimators are unbounded [Pötscher, B.M., 2002. Lower risk bounds and properties of confidence sets for ill-posed estimation problems with applications to spectral density and persistence estimation, unit roots, and estimation of long memory parameters. Econometrica 70, 1035–1065]. We extend these “impossibility results” and show that all tests of size α concerning parameters that can be arbitrarily close to being unidentified have power that can be as small as α for any sample size even if the null and the alternative hypotheses are not adjacent. The results are proved for a very general framework that contains commonly used models.     

Box-Cox变换下联合均值与方差模型的极大似然估计

在提出Box-Cox变换下联合均值与方差模型的基础上,研究了该模型参数的估计问题.同时利用截面极大似然估计方法对变换参数λ进行估计,并对均值模型和方差模型的参数进行极大似然估计.通过随机模拟和实例研究,结果表明该模型和方法是有效和可行的.     

Bootstrap tests for variance components in generalized linear mixed models

In many applications of generalized linear mixed models to clustered correlated or longitudinal data, often we are interested in testing whether a random effects variance component is zero. The usual asymptotic mixture of chi‐square distributions of the score statistic for testing constrained variance components does not necessarily hold. In this article, the author proposes and explores a parametric bootstrap test that appears to be valid based on its estimated level of significance under the null hypothesis. Results from a simulation study indicate that the bootstrap test has a level much closer to the nominal one while the asymptotic test is conservative, and is more powerful than the usual asymptotic score test based on a mixture of chi‐squares. The proposed bootstrap test is illustrated using two sets of real‐life data obtained from clinical trials. The Canadian Journal of Statistics © 2009 Statistical Society of Canada     

On maximum likelihood estimation of the long-memory parameter in fractional Gaussian noise

Approximate normality and unbiasedness of the maximum likelihood estimate (MLE) of the long-memory parameter H of a fractional Brownian motion hold reasonably well for sample sizes as small as 20 if the mean and scale parameter are known. We show in a Monte Carlo study that if the latter two parameters are unknown the bias and variance of the MLE of H both increase substantially. We also show that the bias can be reduced by using a parametric bootstrap procedure. In very large samples, maximum likelihood estimation becomes problematic because of the large dimension of the covariance matrix that must be inverted. To overcome this difficulty, we propose a maximum likelihood method based upon first differences of the data. These first differences form a short-memory process. We split the data into a number of contiguous blocks consisting of a relatively small number of observations. Computation of the likelihood function in a block then presents no computational problem. We form a pseudo-likelihood function consisting of the product of the likelihood functions in each of the blocks and provide a formula for the standard error of the resulting estimator of H. This formula is shown in a Monte Carlo study to provide a good approximation to the true standard error. The computation time required to obtain the estimate and its standard error from large data sets is an order of magnitude less than that required to obtain the widely used Whittle estimator. Application of the methodology is illustrated on two data sets.     

Statistical inference on transformation models: a self-induced smoothing approach

This paper deals with a general class of transformation models that contains many important semiparametric regression models as special cases. It develops a self-induced smoothing for the maximum rank correlation estimator, resulting in simultaneous point and variance estimation. The self-induced smoothing does not require bandwidth selection, yet provides the right amount of smoothness so that the estimator is asymptotically normal with mean zero (unbiased) and variance–covariance matrix consistently estimated by the usual sandwich-type estimator. An iterative algorithm is given for the variance estimation and shown to numerically converge to a consistent limiting variance estimator. The approach is applied to a data set involving survival times of primary biliary cirrhosis patients. Simulation results are reported, showing that the new method performs well under a variety of scenarios.     

Asymptotics for REML estimation of spatial covariance parameters

In agricultural field trials, restricted maximum likelihood estimation (REML) of the spatial covariance parameters is often preferred to maximum likelihood. Although it has either been conjectured or assumed that REML estimators are asymptotically Gaussian, conditions under which such asymptotic results hold are clearly needed. This article gives checkable conditions for spatial regression when sampling locations are either on a rectangular grid or are irregularly spaced but satisfy certain growth conditions.     

A Comparison Between Two Multivariate EWMA Schemes

The signal issued by a control chart triggers the process professionals to investigate the special cause. Change point methods simplify the efforts to search for and identify the special cause. In this study, using maximum likelihood estimation, a multivariate joint change point estimation procedure for monitoring both location and dispersion simultaneously is proposed. After a signal is generated by the simultaneously used Hotelling's T ² and/or generalized variance control charts, the procedure starts detecting the time of the change. The performance of the proposed method for several structural changes for the mean vector and covariance matrix is discussed.     

Goodness‐of‐fit testing based on a weighted bootstrap: A fast large‐sample alternative to the parametric bootstrap

The process comparing the empirical cumulative distribution function of the sample with a parametric estimate of the cumulative distribution function is known as the empirical process with estimated parameters and has been extensively employed in the literature for goodness‐of‐fit testing. The simplest way to carry out such goodness‐of‐fit tests, especially in a multivariate setting, is to use a parametric bootstrap. Although very easy to implement, the parametric bootstrap can become very computationally expensive as the sample size, the number of parameters, or the dimension of the data increase. An alternative resampling technique based on a fast weighted bootstrap is proposed in this paper, and is studied both theoretically and empirically. The outcome of this work is a generic and computationally efficient multiplier goodness‐of‐fit procedure that can be used as a large‐sample alternative to the parametric bootstrap. In order to approximately determine how large the sample size needs to be for the parametric and weighted bootstraps to have roughly equivalent powers, extensive Monte Carlo experiments are carried out in dimension one, two and three, and for models containing up to nine parameters. The computational gains resulting from the use of the proposed multiplier goodness‐of‐fit procedure are illustrated on trivariate financial data. A by‐product of this work is a fast large‐sample goodness‐of‐fit procedure for the bivariate and trivariate t distribution whose degrees of freedom are fixed. The Canadian Journal of Statistics 40: 480–500; 2012 © 2012 Statistical Society of Canada     

D-optimal minimax regression designs on discrete design space

One classical design criterion is to minimize the determinant of the covariance matrix of the regression estimates, and the designs are called D-optimal designs. To reflect the nature that the proposed models are only approximately true, we propose a robust design criterion to study response surface designs. Both the variance and bias are considered in the criterion. In particular, D-optimal minimax designs are investigated and constructed. Examples are given to compare D-optimal minimax designs with classical D-optimal designs.