Missing data in linear models with correlated errors

One common method for analyzing data in experimental designs when observations are missing was devised by Yates (1933), who developed his procedure based upon a suggestion by R. A. Fisher. Considering a linear model with independent, equi-variate errors, Yates substituted algebraic values for the missing data and then minimized the error sum of squares with respect to both the unknown parameters and the algebraic values. Yates showed that this procedure yielded the correct error sum of squares and a positively biased hypothesis sum of squares.

Others have elaborated on this technique. Chakrabarti (1962) gave a formal proof of Fisher's rule that produced a way to simplify the calculations of the auxiliary values to be used in place of the missing observations. Kshirsagar (1971) proved that the hypothesis sum of squares based on these values was biased, and developed an easy way to compute that bias. Sclove     

Extended moment results for improving inferences based on mre'p

The MRPP test statistic studied by Mielke and others is the weighted average distance between pairs of observations within a group. They defined 12 symmetric functions to obtainits first three moments. We define 23 additional symmetric functions to obtain the fourth moment. This can beuseful instudying further approximations to its sampling distribution. We also study the special case when the distance function is the Euclidean distance between ranks of observations     

The probability to select the correct model using likelihood-ratio based criteria in choosing between two nested models of which the more extended one is true

The probability to select the correct model is calculated for likelihood-ratio-based criteria to compare two nested models. If the more extended of the two models is true, the difference between twice the maximised log-likelihoods is approximately noncentral chi-square distributed with d.f. the difference in the number of parameters. The noncentrality parameter of this noncentral chi-square distribution can be approximated by twice the minimum Kullback–Leibler divergence (MKLD) of the best-fitting simple model to the true version of the extended model.The MKLD, and therefore the probability to select the correct model increases approximately proportionally to the number of observations if all observations are performed under the same conditions. If a new set of observations can only be performed under different conditions, the model parameters may depend on the conditions, and therefore have to be estimated for each set of observations separately. An increase in observations will then go together with an increase in the number of model parameters. In this case, the power of the likelihood-ratio test will increase with an increasing number of observations. However, the probability to choose the correct model with the AIC will only increase if for each set of observations the MKLD is more than 0.5. If the MKLD is less than 0.5, that probability will decrease. The probability to choose the correct model with the BIC will always decrease, sometimes after an initial increase for a small number of observation sets. The results are illustrated by a simulation study with a set of five nested nonlinear models for binary data.     

Bayesian analysis of the scatterometer wind retrieval inverse problem: some new approaches

Summary.  The retrieval of wind vectors from satellite scatterometer observations is a non-linear inverse problem. A common approach to solving inverse problems is to adopt a Bayesian framework and to infer the posterior distribution of the parameters of interest given the observations by using a likelihood model relating the observations to the parameters, and a prior distribution over the parameters. We show how Gaussian process priors can be used efficiently with a variety of likelihood models, using local forward (observation) models and direct inverse models for the scatterometer. We present an enhanced Markov chain Monte Carlo method to sample from the resulting multimodal posterior distribution. We go on to show how the computational complexity of the inference can be controlled by using a sparse, sequential Bayes algorithm for estimation with Gaussian processes. This helps to overcome the most serious barrier to the use of probabilistic, Gaussian process methods in remote sensing inverse problems, which is the prohibitively large size of the data sets. We contrast the sampling results with the approximations that are found by using the sparse, sequential Bayes algorithm.     

The effect of Phase I sample size on the run length performance of control charts for autocorrelated data

Traditional control charts assume independence of observations obtained from the monitored process. However, if the observations are autocorrelated, these charts often do not perform as intended by the design requirements. Recently, several control charts have been proposed to deal with autocorrelated observations. The residual chart, modified Shewhart chart, EWMAST chart, and ARMA chart are such charts widely used for monitoring the occurrence of assignable causes in a process when the process exhibits inherent autocorrelation. Besides autocorrelation, one other issue is the unknown values of true process parameters to be used in the control chart design, which are often estimated from a reference sample of in-control observations. Performances of the above-mentioned control charts for autocorrelated processes are significantly affected by the sample size used in a Phase I study to estimate the control chart parameters. In this study, we investigate the effect of Phase I sample size on the run length performance of these four charts for monitoring the changes in the mean of an autocorrelated process, namely an AR(1) process. A discussion of the practical implications of the results and suggestions on the sample size requirements for effective process monitoring are provided.     

Information in an observation in robust designs

We consider the Information contained 1n each observation in a given design robust with respect to the estlmability of parameters and against the unavailability of observations. We compare the observations in various 1-, 2- and 3- dimensional designs on the basis of their informations.     

Mimicking Cox-Regression

Abstract

A class of objective functions, related to the Cox partial likelihood, that generates unbiased estimating equations is proposed. These equations allow for estimation of interest parameters when nuisance parameters are proportional to expectations. Examples of the objective functions are applied to binary data with a log-link in three situations: independent observations, independent groups of observations with common random intercept and discrete survival data. It is pointed out that the Peto–Breslow approximation to the partial likelihood with discrete failure times fits a conditional model with a log-link.     

Continuous single attribute control chart for Markov-dependent processes

Most of the times, the observations related to the quality characteristic of a process do not need to be independent. In such cases, control charts based on the assumption of independence of the observations are not appropriate. When the characteristic under study is qualitative, Markov model serves as a simple model to account for the dependency of the observations. For this purpose, we develop an attribute control chart under 100% inspection for a Markov dependent process by controlling the error probabilities. This chart consists of two sub-charts. For a given sample, depending upon the state of the last observation of previous sample (if any), one of these two will be used. Optimal values of the design parameters of the control chart are obtained. Chart’s performance is studied by using its capability (probability) of detecting a shift in process parameters.     

Bayesian parameter estimation via variational methods

We consider a logistic regression model with a Gaussian prior distribution over the parameters. We show that an accurate variational transformation can be used to obtain a closed form approximation to the posterior distribution of the parameters thereby yielding an approximate posterior predictive model. This approach is readily extended to binary graphical model with complete observations. For graphical models with incomplete observations we utilize an additional variational transformation and again obtain a closed form approximation to the posterior. Finally, we show that the dual of the regression problem gives a latent variable density model, the variational formulation of which leads to exactly solvable EM updates.     

Maximum likelihood estimation in the presence of outiliers

This paper deals with the maximum likelihood estimation of parameters when the sample (x₁…x_n ) may heve k spuriously generated observations from another distribution, say G≠F, where F is the distribution of the target population. If G is stochastically larger than F, then these k observations may give rise to k extreme observations or ‘outliers’. This situation is often described by a so-called ‘k-outlier model’ in which in addition to the parameters involved in F and G, the set ν={ν₁,…,ν_k} of indices, for which x_νj , j=1,…,k, come from G, is also unknow.     

Bayesian sequential inference for nonlinear multivariate diffusions

In this paper, we adapt recently developed simulation-based sequential algorithms to the problem concerning the Bayesian analysis of discretely observed diffusion processes. The estimation framework involves the introduction of m−1 latent data points between every pair of observations. Sequential MCMC methods are then used to sample the posterior distribution of the latent data and the model parameters on-line. The method is applied to the estimation of parameters in a simple stochastic volatility model (SV) of the U.S. short-term interest rate. We also provide a simulation study to validate our method, using synthetic data generated by the SV model with parameters calibrated to match weekly observations of the U.S. short-term interest rate.     

Structural inference for parameters of a power function distribution

We consider the problem of statistical inference on the parameters of the three parameter power function distribution based on a full unordered sample of observations or a type II censored ordered sample of observations. The inference philosophy used is the theory of structural inference. We state inference procedures which yield inferential statements about the three unknown parameters. A numerical example is given to illustrate these procedures. It is seen that within the context of this example the inference procedures of this paper do not encounter certain difficulties associated with classical maximum likelihood based procedures. Indeed it has been our numerical experience that this behavior is typical within the context of that subclass of the three parameter power function distribution to which this example belongs.     

Goodness of fit test for the generalized Rayleigh distribution with unknown parameters

The use of goodness-of-fit test based on Anderson–Darling (AD) statistic is discussed, with reference to the composite hypothesis that a sample of observations comes from a generalized Rayleigh distribution whose parameters are unspecified. Monte Carlo simulation studies were performed to calculate the critical values for AD test. These critical values are then used for testing whether a set of observations follows a generalized Rayleigh distribution when the scale and shape parameters are unspecified and are estimated from the sample. Functional relationship between the critical values of AD is also examined for each shape parameter (α), sample size (n) and significance level (γ). The power study is performed with the hypothesized generalized Rayleigh against alternate distributions.     

Inference about parameters in Binomial-Poisson distribution with additional incomplete data

ABSTRACT

This article considers the distribution of Binomial-Poisson random vector which has two components and includes two parameters: one is the rate of a Poisson distribution, the other is the proportion in a Binomial distribution. The inference about the two parameters is usually made based on only paired observations. However, the number of paired observations is, in general, not large enough because of either technical difficulty or budget limitation, and so one can not make efficient inferences with only paired data. Instead, it is often much easier and not too costly to have incomplete observation on only one component independently. In this article we will combine both the paired complete data and unpaired incomplete data for estimating the two parameters. The performances of various estimators are compared both analytically and numerically. It is observed that fully using the unpaired incomplete data can always improve the inference, and the improvement is very significant in the case when there are only a few paired complete observations.     

GROUPING PROBLEMS IN DISTRIBUTION-FREE REGRESSION1

In regression models having symmetric errors, exact distribution-free inference about individual parameters may be carried out by grouping observations, eliminating unwanted parameters within groups, and applying distribution free techniques for the symmetric location parameter problem. Models whose errors have identical but not symmetric distributions may obtain symmetry by taking differences between pairs of observations. Both grouping and differencing involve potential efficiency loss. The choice of an optimal scheme to minimize efficiency loss is expressible as a multi–assignment type of problem, whose solutions, exact and approximate, are discussed.     

Simultaneous confidence regions for the parameters of damage processes

In connection with the investigation of the reliability of products it is often necessary to consider the development of damage processes of such products to calculate parameters of reliability. The parameters of the damage process can be estimated by observations at discrete time points. If the limit level of damage is known the parameters of life distributions can be calculated by the estimated values of these parameters.     

Testing for heteroskedasticity in the tobit and probit models

Non-constant variance across observations (heteroskedasticity) results in the maximum likelihood estimators of tobit and probit model parameters being inconsistent. Some of the available tests for constant variance across observations (homoskedasticity) are discussed and examined in a small Monte Carlo experiment.     

Improved Estimation of the Pareto Location Parameter Under Squared Error Loss

Suppose two independent observations are drawn from Pareto distributions with known shape parameters and an order restriction on the unknown location parameters. An isotonic regression estimator of the smaller location parameter dominates a preferred marginal estimator under squared error loss, but fails to dominate under stochastic domination. The results expressed herein advance the theory of order restricted inference.     

Robustness of designed experiments against missing data

This paper investigates the robustness of designed experiments for estimating linear functions of a subset of parameters in a general linear model against the loss of any t( ≥1) observations. Necessary and sufficient conditions for robustness of a design under a homoscedastic model are derived. It is shown that a design robust under a homoscedastic model is also robust under a general heteroscedastic model with correlated observations. As a particular case, necessary and sufficient conditions are obtained for the robustness of block designs against the loss of data. Simple sufficient conditions are also provided for the binary block designs to be robust against the loss of data. Some classes of designs, robust up to three missing observations, are identified. A-efficiency of the residual design is evaluated for certain block designs for several patterns of two missing observations. The efficiency of the residual design has also been worked out when all the observations in any two blocks, not necessarily disjoint, are lost. The lower bound to A-efficiency has also been obtained for the loss of t observations. Finally, a general expression is obtained for the efficiency of the residual design when all the observations of m ( ≥1) disjoint blocks are lost.     

Regression with an infinite number of observations applied to estimating the parameters of the stable distribution using the empirical characteristic function

ABSTRACT

The parameters of stable law parameters can be estimated using a regression based approach involving the empirical characteristic function. One approach is to use a fixed number of points for all parameters of the distribution to estimate the characteristic function. In this work the results are derived where all points in an interval is used to estimate the empirical characteristic function, thus least squares estimators of a linear function of the parameters, using an infinite number of observations. It was found that the procedure performs very good in small samples.