An implicit function based procedure for analyzing maximum likelihood estimates from nonidentically distributed data

A methodology is presented for gaining insight into properties — such as outlier influence, bias, and width of confidence intervals — of maximum likelihood estimates from nonidentically distributed Gaussian data. The methodology is based on an application of the implicit function theorem to derive an approximation to the maximum likelihood estimator. This approximation, unlike the maximum likelihood estimator, is expressed in closed form and thus it can be used in lieu of costly Monte Carlo simulation to study the properties of the maximum likelihood estimator.     

New Indices of Money Supply and the Flexible Laurent Demand System

The article begins by surveying the existing results on the new Divisia monetary aggregates. Charts display the differences in behavior between the Divisia aggregates and the Federal Reserve's official simple-sum monetary aggregates. The article then compares system-wide fit for the simple-sum and Divisia monetary aggregates when used as data in the joint estimation of a system of demand equations. The demand system is derived from a new Laurent expansion approximation to the reciprocal indirect utility function. The Laurent expansion provides a better-behaved remainder term than that of the more commonly used Taylor series. The results favor the Divisia aggregates.     

The lognormal characteristic function

A number of different ways are examined of representing the characteristic function φ(t) of the lognormal distribution, which cannot be expanded in a Taylor series based on the moments. In §2 the use of a finite Taylor series is examined. A method of summing the divergent formal expansion is discussed in §3. In §4 the fact that φ(t) is a boundary analytic function is exploited. Asymptotic approximation of the integral defining φ(t) is studied in §5. Each approach produces some interesting information about the distribution.     

Second-order variance estimation in poststratified two-stage sampling

The linearization or Taylor series variance estimator and jackknife linearization variance estimator are popular for poststratified point estimators. In this note we propose a simple second-order linearization variance estimator for the poststratified estimator of the population total in two-stage sampling, using the second-order Taylor series expansion. We investigate the properties of the proposed variance estimator and its modified version and their empirical performance through some simulation studies in comparison to the standard and jackknife linearization variance estimators. Simulation studies are carried out on both artificially generated data and real data.     

Approximating Matrix-Exponential Distributions by Global Randomization

Abstract

Based on the general concept of randomization, we develop linear-algebraic approximations for continuous probability distributions that involve the exponential of a matrix in their definitions, such as phase types and matrix-exponential distributions. The approximations themselves result in proper probability distributions. For such a global randomization with the Erlang-k distribution, we show that the sequences of true and consistent distribution and density functions converge uniformly on [0, ∞). Furthermore, we study the approximation errors in terms of the power moments and the coefficients of the Taylor series, from which the accuracy of the approximations can be determined apriori. Numerical experiments demonstrate the feasibility of the presented randomization technique – also in comparison with uniformization.     

Estimation of generalized linear latent variable models

Summary.  Generalized linear latent variable models (GLLVMs), as defined by Bartholomew and Knott, enable modelling of relationships between manifest and latent variables. They extend structural equation modelling techniques, which are powerful tools in the social sciences. However, because of the complexity of the log-likelihood function of a GLLVM, an approximation such as numerical integration must be used for inference. This can limit drastically the number of variables in the model and can lead to biased estimators. We propose a new estimator for the parameters of a GLLVM, based on a Laplace approximation to the likelihood function and which can be computed even for models with a large number of variables. The new estimator can be viewed as an M -estimator, leading to readily available asymptotic properties and correct inference. A simulation study shows its excellent finite sample properties, in particular when compared with a well-established approach such as LISREL. A real data example on the measurement of wealth for the computation of multidimensional inequality is analysed to highlight the importance of the methodology.     

A general approximation to quantiles

For many continuous distributions, a closed-form expression for their quantiles does not exist. Numerical approximations for their quantiles are developed on a distribution-by-distribution basis. This work develops a general approximation for quantiles using the Taylor expansion. Our method only requires that the distribution has a continuous probability density function and its derivatives can be derived to a certain order (usually 3 or 4). We demonstrate our unified approach by approximating the quantiles of the normal, exponential, and chi-square distributions. The approximation works well for these distributions.     

DISTRIBUTION OF THE LEAST SQUARES ESTIMATOR IN A FIRST-ORDER AUTOREGRESSIVE MODEL

This paper investigates the finite sample distribution of the least squares estimator of the autoregressive parameter in a first-order autoregressive model. A uniform asymptotic expansion for the distribution applicable to both stationary and nonstationary cases is obtained. Accuracy of the approximation to the distribution by a first few terms of this expansion is then investigated. It is found that the leading term of this expansion approximates well the distribution. The approximation is, in almost all cases, accurate to the second decimal place throughout the distribution. In the literature, there exist a number of approximations to this distribution which are specifically designed to apply in some special cases of this model. The present approximation compares favorably with those approximations and in fact, its accuracy is, with almost no exception, as good as or better than these other approximations. Convenience of numerical computations seems also to favor the present approximations over the others. An application of the finding is illustrated with examples.     

Approximation of distributions by using the Anderson Darling statistic

ABSTRACT

In practice, it is often not possible to find an appropriate family of distributions which can be used for fitting the sample distribution with high precision. In these cases, it seems to be opportune to search for the best approximation by a family of distributions instead of an exact fit. In this paper, we consider the Anderson–Darling statistic with plugged-in minimum distance estimator for the parameter vector. We prove asymptotic normality of the Anderson–Darling statistic which is used for a test of goodness of approximation. Moreover, we introduce a measure of discrepancy between the sample distribution and the model class.     

An approximation for the hotelling-lawley trace in the noncentral case

A procedure for estimating power in conjunction with the Hotelling-Lawley trace is developed. By approximating a non-central Wishart distribution with a central Wishart, and using McKeon's (1974) F-type approximation, a relatively simple procedure for obtaining power estimates is obtained. The accuracy of the approximation is investigated by comparing the approximate results with those for a wide range of conditions given in Olson's (1973) extensive Monte Carlo study. Siotani's (1971) asymptotic expansion is used to provide further comparative assessments. It is demonstrated that the approximation is of sufficient accuracy to be used in practical applications.     

Bayes Estimation of Intraclass Correlation Coefficients Under Unequal Family Sizes

Bayesian inference for the intraclass correlation ρ is considered under unequal family sizes. We obtain the posterior distribution of ρ and then compare the performance of the Bayes estimator (posterior mean of ρ) with that of Srivastava's (1984) estimator through simulation. Simulation study shows that the Bayes estimator performs better than the Srivastava's estimator in terms of lower mean square error. We also obtain large sample posteriors of ρ based on the asymptotic posterior distribution and based on the Laplace approximation.     

Limited information likelihood analysis of survey data

Analysts of survey data are often interested in modelling the population process, or superpopulation, that gave rise to a 'target' set of survey variables. An important tool for this is maximum likelihood estimation. A survey is said to provide limited information for such inference if data used in the design of the survey are unavailable to the analyst. In this circumstance, sample inclusion probabilities, which are typically available, provide information which needs to be incorporated into the analysis. We consider the case where these inclusion probabilities can be modelled in terms of a linear combination of the design and target variables, and only sample values of these are available. Strict maximum likelihood estimation of the underlying superpopulation means of these variables appears to be analytically impossible in this case, but an analysis based on approximations to the inclusion probabilities leads to a simple estimator which is a close approximation to the maximum likelihood estimator. In a simulation study, this estimator outperformed several other estimators that are based on approaches suggested in the sampling literature.     

Calibrated imputation in surveys under a quasi-model-assisted approach

Summary.  We propose to use calibrated imputation to compensate for missing values. This technique consists of finding final imputed values that are as close as possible to preliminary imputed values and are calibrated to satisfy constraints. Preliminary imputed values, potentially justified by an imputation model, are obtained through deterministic single imputation. Using appropriate constraints, the resulting imputed estimator is asymptotically unbiased for estimation of linear population parameters such as domain totals. A quasi-model-assisted approach is considered in the sense that inferences do not depend on the validity of an imputation model and are made with respect to the sampling design and a non-response model. An imputation model may still be used to generate imputed values and thus to improve the efficiency of the imputed estimator. This approach has the characteristic of handling naturally the situation where more than one imputation method is used owing to missing values in the variables that are used to obtain imputed values. We use the Taylor linearization technique to obtain a variance estimator under a general non-response model. For the logistic non-response model, we show that ignoring the effect of estimating the non-response model parameters leads to overestimating the variance of the imputed estimator. In practice, the overestimation is expected to be moderate or even negligible, as shown in a simulation study.     

The stochastic approximation method for the estimation of a multivariate probability density

We apply the stochastic approximation method to construct a large class of recursive kernel estimators of a probability density, including the one introduced by Hall and Patil [1994. On the efficiency of on-line density estimators. IEEE Trans. Inform. Theory 40, 1504–1512]. We study the properties of these estimators and compare them with Rosenblatt's nonrecursive estimator. It turns out that, for pointwise estimation, it is preferable to use the nonrecursive Rosenblatt's kernel estimator rather than any recursive estimator. A contrario, for estimation by confidence intervals, it is better to use a recursive estimator rather than Rosenblatt's estimator.     

COMBINING INDIVIDUAL PARTICIPANT DATA AND SUMMARY STATISTICS FROM BOTH CONTINUOUSLY VALUED AND BINARY VARIABLES TO ESTIMATE REGRESSION PARAMETERS

Recent research has extended standard methods for meta‐analysis to more general forms of evidence synthesis, where the aim is to combine different data types or data summaries that contain information about functions of multiple parameters to make inferences about the parameters of interest. We consider one such scenario in which the goal is to make inferences about the association between a primary binary exposure and continuously valued outcome in the context of several confounding exposures, and where the data are available in various different forms: individual participant data (IPD) with repeated measures, sample means that have been aggregated over strata, and binary data generated by thresholding the underlying continuously valued outcome measure. We show that an estimator of the population mean of a continuously valued outcome can be constructed using binary threshold data provided that a separate estimate of the outcome standard deviation is available. The results of a simulation study show that this estimator has negligible bias but is less efficient than the sample mean – the minimum variance ratio is based on a Taylor series expansion. Combining this estimator with sample means and IPD from different sources (such as a series of published studies) using both linear and probit regression does, however, improve the precision of estimation considerably by incorporating data that would otherwise have been excluded for being in the wrong format. We apply these methods to investigate the association between the G277S mutation in the transferrin gene and serum ferritin (iron) levels separately in pre‐ and post‐menopausal women based on data from three published studies.     

Exact and Approximate Statistical Inference for Nonlinear Regression and the Estimating Equation Approach

The exact density distribution of the non‐linear least squares estimator in the one‐parameter regression model is derived in closed form and expressed through the cumulative distribution function of the standard normal variable. Several proposals to generalize this result are discussed. The exact density is extended to the estimating equation (EE) approach and the non‐linear regression with an arbitrary number of linear parameters and one intrinsically non‐linear parameter. For a very special non‐linear regression model, the derived density coincides with the distribution of the ratio of two normally distributed random variables previously obtained by Fieler almost a century ago, unlike other approximations previously suggested by other authors. Approximations to the density of the EE estimators are discussed in the multivariate case. Numerical complications associated with the non‐linear least squares are illustrated, such as non‐existence and/or multiple solutions, as major factors contributing to poor density approximation. The non‐linear Markov–Gauss theorem is formulated on the basis of the near exact EE density approximation.     

Simple and accurate inference for the mean of the gamma model

The two-parameter gamma model is widely used in reliability, environmental, medical and other areas of statistics. It has a two-dimensional sufficient statistic, and a two-dimensional parameter which can be taken to describe shape and mean. This makes it closely comparable to the normal model, but it differs substantially in that the exact distribution for the minimal sufficient statistic is not available. Some recently developed asymptotic theory is used to derive an approximation for observed levels of significance and confidence intervals for the mean parameter of the model. The approximation is as easy to apply as first-order methods, and substantially more accurate.     

On the ridge regression estimator with sub-space restriction

In the linear regression model with elliptical errors, a shrinkage ridge estimator is proposed. In this regard, the restricted ridge regression estimator under sub-space restriction is improved by incorporating a general function which satisfies Taylor’s series expansion. Approximate quadratic risk function of the proposed shrinkage ridge estimator is evaluated in the elliptical regression model. A Monte Carlo simulation study and analysis based on a real data example are considered for performance analysis. It is evident from the numerical results that the shrinkage ridge estimator performs better than both unrestricted and restricted estimators in the multivariate t-regression model, for some specific cases.     

Pivotal inference for the inverse Rayleigh distribution based on general progressively Type-II censored samples

In this paper, we consider the problem of estimating the scale parameter of the inverse Rayleigh distribution based on general progressively Type-II censored samples and progressively Type-II censored samples. The pivotal quantity method is used to derive the estimator of the scale parameter. Besides, considering that the maximum likelihood estimator is tough to obtain for this distribution, we derive an explicit estimator of the scale parameter by approximating the likelihood equation with Taylor expansion. The interval estimation is also studied based on pivotal inference. Then we conduct Monte Carlo simulations and compare the performance of different estimators. We demonstrate that the pivotal inference is simpler and more effective. The further application of the pivotal quantity method is also discussed theoretically. Finally, two real data sets are analyzed using our methods.     

Estimation and Inference Based on Neumann Series Approximation to Locally Efficient Score in Missing Data Problems

Abstract.  Theory on semi-parametric efficient estimation in missing data problems has been systematically developed by Robins and his coauthors. Except in relatively simple problems, semi-parametric efficient scores cannot be expressed in closed forms. Instead, the efficient scores are often expressed as solutions to integral equations. Neumann series was proposed in the form of successive approximation to the efficient scores in those situations. Statistical properties of the estimator based on the Neumann series approximation are difficult to obtain and as a result, have not been clearly studied. In this paper, we reformulate the successive approximation in a simple iterative form and study the statistical properties of the estimator based on the reformulation. We show that a doubly robust locally efficient estimator can be obtained following the algorithm in robustifying the likelihood score. The results can be applied to, among others, parametric regression, marginal regression and Cox regression when data are subject to missing values and the data are missing at random. A simulation study is conducted to evaluate the performance of the approach and a real data example is analysed to demonstrate the use of the approach.