Sequential determination of the number of bootstrap samples

The primary purpose of this paper is that of developing a sequential Monte Carlo approximation to an ideal bootstrap estimate of the parameter of interest. Using the concept of fixed-precision approximation, we construct a sequential stopping rule for determining the number of bootstrap samples to be taken in order to achieve a specified precision of the Monte Carlo approximation. It is shown that the sequential Monte Carlo approximation is asymptotically efficient in the problems of estimation of the bias and standard error of a given statistic. Efficient bootstrap resampling is discussed and a numerical study is carried out for illustrating the obtained theoretical results.     

Solving the Stochastic Growth Model by Backsolving With a Particular Nonlinear Form for the Decision Rule

Backsolving is a class of methods that generate simulated values for exogenous forcing processes in a stochastic equilibrium model from specified assumed distributions for Euler-equation disturbances. It can be thought of as a way to force the approximation error generated by inexact choice of decision rule or boundary condition into distortions of the distribution of the exogenous shocks in the simulations rather than into violations of the Euler equations as with standard approaches. Here it is applied to a one-sector neoclassical growth model with decision rule generated from a linear-quadratic approximation.     

An iterative sparse algorithm for the penalized maximum likelihood estimator in mixed effects model

In this paper, we propose a new iterative sparse algorithm (ISA) to compute the maximum likelihood estimator (MLE) or penalized MLE of the mixed effects model. The sparse approximation based on the arrow-head (A-H) matrix is one solution which is popularly used in practice. The A-H method provides an easy computation of the inverse of the Hessian matrix and is computationally efficient. However, it often has non-negligible error in approximating the inverse of the Hessian matrix and in the estimation. Unlike the A-H method, in the ISA, the sparse approximation is applied “iteratively” to reduce the approximation error at each Newton Raphson step. The advantages of the ISA over the exact and A-H method are illustrated using several synthetic and real examples.     

Likelihood inference for dynamic linear models with Markov switching parameters: on the efficiency of the Kim filter

The Kim filter (KF) approximation is widely used for the likelihood calculation of dynamic linear models with Markov regime-switching parameters. However, despite its popularity, its approximation error has not yet been examined rigorously. Therefore, this study investigates the reliability of the KF approximation for maximum likelihood (ML) and Bayesian estimations. To measure the approximation error, we compare the outcomes of the KF method with those of the auxiliary particle filter (APF). The APF is a numerical method that requires a longer computing time, but its numerical error can be sufficiently minimized by increasing simulation size. According to our extensive simulation and empirical studies, the likelihood values obtained from the KF approximation are practically identical to those of the APF. Furthermore, we show that the KF method is reliable, particularly when regimes are persistent and sample size is small. From the Bayesian perspective, we show that the KF method improves the efficiency of posterior simulation. This study contributes to the literature by providing evidence to justify the use of the KF method in both ML and Bayesian estimations.     

Euler characteristic heuristic for approximating the distribution of the largest eigenvalue of an orthogonally invariant random matrix

The Euler characteristic heuristic has been proposed as a method for approximating the upper tail probability of the maximum of a random field with smooth sample path. When the random field is Gaussian, this method is proved to be valid in the sense that the relative approximation error is exponentially smaller. However, very little is known about the validity of the method when the random field is non-Gaussian. In this paper, as a milestone to developing the general theory about the validity of the Euler characteristic heuristic, we examine the Euler characteristic heuristic for approximating the distribution of the largest eigenvalue of an orthogonally invariant non-Gaussian random matrix. In this particular example, if the probability density function of the random matrix converges to zero sufficiently fast at the boundary of its support, the approximation error of the Euler characteristic heuristic is proved to be small and the approximation is valid. Moreover, for several standard orthogonally invariant random matrices, the approximation formula for the distribution of the largest eigenvalue and its asymptotic error are obtained explicitly. Our formulas are practical enough for the purpose of numerical calculations.     

Estimation of error rate in multiple group logistic discrimination. the approximate leaving-one-out method

We present an approximate leaving-one-out technique for estimating the error rate in logistic discrimination. The new measure is based on the one-step approximation of a(i), the maximum likelihood estimate of the parameter vector based on the sample without the ith case. Some inequalities between the resubstitution error rate, the approximate and exact leaving-one-out error rates for the multiple group logistic model are investigated. Monte-Carlo simulations assess the adequacy of the approximate leaving-one-out method as an estimate of the actual error rate. The usefulness of this approach is demonstrated by means of two medical examples.     

Estimation for the Parameters of Generalized Half-normal Distribution Based on Progressive Type-I Interval Censoring

Based on progressively Type-I interval censored sample, the problem of estimating unknown parameters of a two parameter generalized half-normal(GHN) distribution is considered. Different methods of estimation are discussed. They include the maximum likelihood estimation, midpoint approximation method, approximate maximum likelihood estimation, method of moments, and estimation based on probability plot. Several Bayesian estimates with respect to different symmetric and asymmetric loss functions such as squared error, LINEX, and general entropy is calculated. The Lindley’s approximation method is applied to determine Bayesian estimates. Monte Carlo simulations are performed to compare the performances of the different methods. Finally, analysis is also carried out for a real dataset.     

Estimation bias and bias correction in reduced rank autoregressions

This paper characterizes the finite-sample bias of the maximum likelihood estimator (MLE) in a reduced rank vector autoregression and suggests two simulation-based bias corrections. One is a simple bootstrap implementation that approximates the bias at the MLE. The other is an iterative root-finding algorithm implemented using stochastic approximation methods. Both algorithms are shown to be improvements over the MLE, measured in terms of mean square error and mean absolute deviation. An illustration to US macroeconomic time series is given.     

A Modular CDF Approach for the Approximation of Percentiles

This article describes a method for computing approximate statistics for large data sets, when exact computations may not be feasible. Such situations arise in applications such as climatology, data mining, and information retrieval (search engines). The key to our approach is a modular approximation to the cumulative distribution function (cdf) of the data. Approximate percentiles (as well as many other statistics) can be computed from this approximate cdf. This enables the reduction of a potentially overwhelming computational exercise into smaller, manageable modules. We illustrate the properties of this algorithm using a simulated data set. We also examine the approximation characteristics of the approximate percentiles, using a von Mises functional type approach. In particular, it is shown that the maximum error between the approximate cdf and the actual cdf of the data is never more than 1% (or any other preset level). We also show that under assumptions of underlying smoothness of the cdf, the approximation error is much lower in an expected sense. Finally, we derive bounds for the approximation error of the percentiles themselves. Simulation experiments show that these bounds can be quite tight in certain circumstances.     

Maximizing generalized linear mixed model likelihoods with an automated Monte Carlo EM algorithm

Two new implementations of the EM algorithm are proposed for maximum likelihood fitting of generalized linear mixed models. Both methods use random (independent and identically distributed) sampling to construct Monte Carlo approximations at the E-step. One approach involves generating random samples from the exact conditional distribution of the random effects (given the data) by rejection sampling, using the marginal distribution as a candidate. The second method uses a multivariate t importance sampling approximation. In many applications the two methods are complementary. Rejection sampling is more efficient when sample sizes are small, whereas importance sampling is better with larger sample sizes. Monte Carlo approximation using random samples allows the Monte Carlo error at each iteration to be assessed by using standard central limit theory combined with Taylor series methods. Specifically, we construct a sandwich variance estimate for the maximizer at each approximate E-step. This suggests a rule for automatically increasing the Monte Carlo sample size after iterations in which the true EM step is swamped by Monte Carlo error. In contrast, techniques for assessing Monte Carlo error have not been developed for use with alternative implementations of Monte Carlo EM algorithms utilizing Markov chain Monte Carlo E-step approximations. Three different data sets, including the infamous salamander data of McCullagh and Nelder, are used to illustrate the techniques and to compare them with the alternatives. The results show that the methods proposed can be considerably more efficient than those based on Markov chain Monte Carlo algorithms. However, the methods proposed may break down when the intractable integrals in the likelihood function are of high dimension.     

A consistency result in general censoring models

In this paper we prove a consistency result for sieved maximum likelihood estimators of the density in general random censoring models with covariates. The proof is based on the method of functional estimation. The estimation error is decomposed in a deterministic approximation error and the stochastic estimation error. The main part of the proof is to establish a uniform law of large numbers for the conditional log-likelihood functional, by using results and techniques from empirical process theory.     

Asymptotic relative efficiency of the linear discriminant function under partial nonrandom classification of the training data

This paper considers the problem where the linear discriminant rule is formed from training data that are only partially classified with respect to the two groups of origin. A further complication is that the data of unknown origin do not constitute an observed random sample from a mixture of the two under- lying groups. Under the assumption of a homoscedastic normal model, the overall error rate of the sample linear discriminant rule formed by maximum likelihood from the partially classified training data is derived up to and including terms of the first order in the case of univariate feature data. This first- order expansion of the sample rule so formed is used to define its asymptotic efficiency relative to the rule formed from a completely classified random training set and also to the rule formed from a completely unclassified random set.     

Estimating the parameters of a Burr distribution under progressive type II censoring

The problem of estimating unknown parameters and reliability function of a two parameter Burr type XII distribution is considered on the basis of a progressively type II censored sample. Several Bayesian estimates are obtained against different symmetric and asymmetric loss functions such as squared error, linex and general entropy. These Bayesian estimates are evaluated by applying the Lindley approximation method. Using simulations, all Bayesian estimates are compared with the corresponding maximum likelihood estimates numerically in terms of their mean square error values and some specific comments are made. Finally, two data sets are analyzed for the purpose of illustration.     

Some Analytical and Numerical Bounds on the Renewal Function

Renewal-type equations are frequently encountered in the study of reliability, warranty analysis, replacement and maintenance policies, and inventory control. Renewal equations usually do not have analytical solutions, and hence, bounds or approximations are very useful. In this article, analytical bounds are studied based on a simple iterative procedure which provides some analytical results and nice convergence properties when the number of iteration increases. Bounds and approximations are also investigated for a recursive algorithm for numerical computation. In addition, some interesting monotonicity properties are introduced and discussed. The approximation error, which is important for determining the stopping rule of the iterative procedure and the numerical algorithm, is also studied.     

Empirical distribution function under heteroscedasticity

Neglecting heteroscedasticity of error terms may imply the wrong identification of a regression model (see appendix). Employment of (heteroscedasticity resistent) White's estimator of covariance matrix of estimates of regression coefficients may lead to the correct decision about the significance of individual explanatory variables under heteroscedasticity. However, White's estimator of covariance matrix was established for least squares (LS)-regression analysis (in the case when error terms are normally distributed, LS- and maximum likelihood (ML)-analysis coincide and hence then White's estimate of covariance matrix is available for ML-regression analysis, tool). To establish White's-type estimate for another estimator of regression coefficients requires Bahadur representation of the estimator in question, under heteroscedasticity of error terms. The derivation of Bahadur representation for other (robust) estimators requires some tools. As the key too proved to be a tight approximation of the empirical distribution function (d.f.) of residuals by the theoretical d.f. of the error terms of the regression model. We need the approximation to be uniform in the argument of d.f. as well as in regression coefficients. The present paper offers this approximation for the situation when the error terms are heteroscedastic.     

A tobit model with garch errors

In the context of time series regression, we extend the standard Tobit model to allow for the possibility of conditional heteroskedastic error processes of the GARCH type. We discuss the likelihood function of the Tobit model in the presence of conditionally heteroskedastic errors. Expressing the exact likelihood function turns out to be infeasible, and we propose an approximation by treating the model as being conditionally Gaussian. The performance of the estimator is investigated by means of Monte Carlo simulations. We find that, when the error terms follow a GARCH process, the proposed estimator considerably outperforms the standard Tobit quasi maximum likelihood estimator. The efficiency loss due to the approximation of the likelihood is finally evaluated.     

A new approximation to the distribution function of the studentized maximum root

The Studentized maximum root (SMR) distribution is useful for constructing simultaneous confidence intervals around product interaction contrasts in replicated two-way ANOVA. A three-moment approximation to the SMR distribution is proposed. The approximation requires the first three moments of the maximum root of a central Wishart matrix. These values are obtained by means of numerical integration. The accuracy of the approximation is compared to the accuracy of a two-moment approximation for selected two-way table sizes. Both approximations are reasonably accurate. The three-moment approximation is generally superior.     

Robustness of the linear discriminant function to nonnormality: Edgeworth series distribution

The effects of applying the normal classificatory rule to a nonnormal population are studied here. These are assessed through the distribution of the misclassification errors in the case of the Edgeworth type distribution. Both theoretical and empirical results are presented. An examination of the latter shows that the effects of this type of nonnormality are marginal. The probability of misclassification of an observation from ∏₁, using the appropriate LR rule, is always larger than one using the normal approximation (μ₁<μ₂). Converse condition holds for the misclassification of an observation from ∏₂. Overall error rates are not affected by the skewness factor to any great extent.     

An automated (Markov chain) Monte Carlo EM algorithm

We present an automated Monte Carlo EM (MCEM) algorithm which efficiently assesses Monte Carlo error in the presence of dependent Monte Carlo, particularly Markov chain Monte Carlo, E-step samples and chooses an appropriate Monte Carlo sample size to minimize this Monte Carlo error with respect to progressive EM step estimates. Monte Carlo error is gauged though an application of the central limit theorem during renewal periods of the MCMC sampler used in the E-step. The resulting normal approximation allows us to construct a rigorous and adaptive rule for updating the Monte Carlo sample size each iteration of the MCEM algorithm. We illustrate our automated routine and compare the performance with competing MCEM algorithms in an analysis of a data set fit by a generalized linear mixed model.     

Computation of the asymptotic null distribution of goodness-of-fit tests for multi-state models

We develop an improved approximation to the asymptotic null distribution of the goodness-of-fit tests for panel observed multi-state Markov models (Aguirre-Hernandez and Farewell, Stat Med 21:1899–1911, 2002) and hidden Markov models (Titman and Sharples, Stat Med 27:2177–2195, 2008). By considering the joint distribution of the grouped observed transition counts and the maximum likelihood estimate of the parameter vector it is shown that the distribution can be expressed as a weighted sum of independent c²₁{\chi^2_1} random variables, where the weights are dependent on the true parameters. The performance of this approximation for finite sample sizes and where the weights are calculated using the maximum likelihood estimates of the parameters is considered through simulation. In the scenarios considered, the approximation performs well and is a substantial improvement over the simple χ ² approximation.