Bayesian Analysis in Regression Models Using Pseudo-Likelihoods

This article deals with the issue of using a suitable pseudo-likelihood, instead of an integrated likelihood, when performing Bayesian inference about a scalar parameter of interest in the presence of nuisance parameters. The proposed approach has the advantages of avoiding the elicitation on the nuisance parameters and the computation of multidimensional integrals. Moreover, it is particularly useful when it is difficult, or even impractical, to write the full likelihood function.

We focus on Bayesian inference about a scalar regression coefficient in various regression models. First, in the context of non-normal regression-scale models, we give a theroetical result showing that there is no loss of information about the parameter of interest when using a posterior distribution derived from a pseudo-likelihood instead of the correct posterior distribution. Second, we present non trivial applications with high-dimensional, or even infinite-dimensional, nuisance parameters in the context of nonlinear normal heteroscedastic regression models, and of models for binary outcomes and count data, accounting also for possibile overdispersion. In all these situtations, we show that non Bayesian methods for eliminating nuisance parameters can be usefully incorporated into a one-parameter Bayesian analysis.     

Control variates and importance sampling for efficient bootstrap simulations

Importance sampling and control variates have been used as variance reduction techniques for estimating bootstrap tail quantiles and moments, respectively. We adapt each method to apply to both quantiles and moments, and combine the methods to obtain variance reductions by factors from 4 to 30 in simulation examples.We use two innovations in control variates—interpreting control variates as a re-weighting method, and the implementation of control variates using the saddlepoint; the combination requires only the linear saddlepoint but applies to general statistics, and produces estimates with accuracy of order n ^-1/2 B ^-1, where n is the sample size and B is the bootstrap sample size.We discuss two modifications to classical importance sampling—a weighted average estimate and a mixture design distribution. These modifications make importance sampling robust and allow moments to be estimated from the same bootstrap simulation used to estimate quantiles.     

MODELING AND ESTIMATING VARIANCES IN REGRESSION

ABSTRACT

It is well known that ignoring heteroscedasticity in regression analysis adversely affects the efficiency of estimation and renders the usual procedure for constructing prediction intervals inappropriate. In some applications, such as off-line quality control, knowledge of the variance function is also of considerable interest in its own right. Thus the modeling of variance constitutes an important part of regression analysis. A common practice in modeling variance is to assume that a certain function of the variance can be closely approximated by a function of a known parametric form. The logarithm link function is often used even if it does not fit the observed variation satisfactorily, as other alternatives may yield negative estimated variances. In this paper we propose a rich class of link functions for more flexible variance modeling which alleviates the major difficulty of negative variances. We suggest also an alternative analysis for heteroscedastic regression models that exploits the principle of “separation” discussed in Box (Signal-to-Noise Ratios, Performance Criteria and Transformation. Technometrics 1988, 30, 1–31). The proposed method does not require any distributional assumptions once an appropriate link function for modeling variance has been chosen. Unlike the analysis in Box (Signal-to-Noise Ratios, Performance Criteria and Transformation. Technometrics 1988, 30, 1–31), the estimated variances and their associated asymptotic variances are found in the original metric (although a transformation has been applied to achieve separation in a different scale), making interpretation of results considerably easier.     

Variance function in regression analysis of longitudinal data using the generalized estimating equation approach

Longitudinal or clustered response data arise in many applications such as biostatistics, epidemiology and environmental studies. The repeated responses cannot in general be assumed to be independent. One method of analysing such data is by using the generalized estimating equations (GEE) approach. The current GEE method for estimating regression effects in longitudinal data focuses on the modelling of the working correlation matrix assuming a known variance function. However, correct choice of the correlation structure may not necessarily improve estimation efficiency for the regression parameters if the variance function is misspecified [Wang YG, Lin X. Effects of variance-function misspecification in analysis of longitudinal data. Biometrics. 2005;61:413–421]. In this connection two problems arise: finding a correct variance function and estimating the parameters of the chosen variance function. In this paper, we study the problem of estimating the parameters of the variance function assuming that the form of the variance function is known and then the effect of a misspecified variance function on the estimates of the regression parameters. We propose a GEE approach to estimate the parameters of the variance function. This estimation approach borrows the idea of Davidian and Carroll [Variance function estimation. J Amer Statist Assoc. 1987;82:1079–1091] by solving a nonlinear regression problem where residuals are regarded as the responses and the variance function is regarded as the regression function. A limited simulation study shows that the proposed method performs at least as well as the modified pseudo-likelihood approach developed by Wang and Zhao [A modified pseudolikelihood approach for analysis of longitudinal data. Biometrics. 2007;63:681–689]. Both these methods perform better than the GEE approach.     

Bayesian approach to errors-in-variables in count data regression models with departures from normality and overdispersion

In most practical applications, the quality of count data is often compromised due to errors-in-variables (EIVs). In this paper, we apply Bayesian approach to reduce bias in estimating the parameters of count data regression models that have mismeasured independent variables. Furthermore, the exposure model is misspecified with a flexible distribution, hence our approach remains robust against any departures from normality in its true underlying exposure distribution. The proposed method is also useful in realistic situations as the variance of EIVs is estimated instead of assumed as known, in contrast with other methods of correcting bias especially in count data EIVs regression models. We conduct simulation studies on synthetic data sets using Markov chain Monte Carlo simulation techniques to investigate the performance of our approach. Our findings show that the flexible Bayesian approach is able to estimate the values of the true regression parameters consistently and accurately.     

One-cluster sampling designs for an autocorrelated population

The variance of the sampling distribution of the sample mean is derived for two sampling designs in which a single cluster is randomly drawn from an autocorrelated population. The derivations are motivated by potential applications to statistical quality control, where a "one-cluster" sampling design may often be used because of ease of implementation, and where it is likely that process output is autocorrelated Scenarios in statistical process control for which either non-overlapping or overlapping clusters are appropriate are described The sampling design variance under non-overlapping clusters is related to the sampling design variance under overlapping clusters through the use of a circular population.     

Design and relative efficiency in two-phase studies

Many two-phase sampling designs have been applied in practice to obtain efficient estimates of regression parameters while minimizing the cost of data collection. This research investigates two-phase sampling designs for so-called expensive variable problems, and compares them with one-phase designs. Closed form expressions for the asymptotic relative efficiency of maximum likelihood estimators from the two designs are derived for parametric normal models, providing insight into the available information for regression coefficients under the two designs. We further discuss when we should apply the two-phase design and how to choose the sample sizes for two-phase samples. Our numerical study indicates that the results can be applied to more general settings.     

Modelling udder infection data using copula models for quadruples

We study copula models for correlated infection times in the four udder quarters of dairy cows. Both a semi-parametric and a nonparametric approach are considered to estimate the marginal survival functions, taking into account the effect of a binary udder quarter level covariate. We use a two-stage estimation approach and we briefly discuss the asymptotic behaviour of the estimators obtained in the first and the second stage of the estimation. A pseudo-likelihood ratio test is used to select an appropriate copula from the power variance copula family that describes the association between the outcomes in a cluster. We propose a new bootstrap algorithm to obtain the p-value for this test. This bootstrap algorithm also provides estimates for the standard errors of the estimated parameters in the copula. The proposed methods are applied to the udder infection data. A small simulation study for a setting similar to the setting of the udder infection data gives evidence that the proposed method provides a valid approach to select an appropriate copula within the power variance copula family.     

Adjustment of the Profile Likelihood for a Class of Normal Regression Models

We apply the method of McCullagh & Tibshirani (1990) to a generalization of the model for variance components in which the parameter of interest can appear in both the mean and variance. We obtain the exact adjusted profile log-likelihood score function. For the variance components model, we obtain the adjusted profile log-likelihood and show that it equals the restricted log-likelihood of Patterson & Thompson (1971). We discuss an example due to Kempton (1982) of a regression model with autoregressive terms in which the parameter of interest appears in both the mean and variance.     

The Effect of Methods for Handling Missing Values on the Performance of the MEWMA Control Chart

This article is concerned with the effect of the methods for handling missing values in multivariate control charts. We discuss the complete case, mean substitution, regression, stochastic regression, and the expectation–maximization algorithm methods for handling missing values. Estimates of mean vector and variance–covariance matrix from the treated data set are used to build the multivariate exponentially weighted moving average (MEWMA) control chart. Based on a Monte Carlo simulation study, the performance of each of the five methods is investigated in terms of its ability to obtain the nominal in-control and out-of-control average run length (ARL). We consider three sample sizes, five levels of the percentage of missing values, and three types of variable numbers. Our simulation results show that imputation methods produce better performance than case deletion methods. The regression-based imputation methods have the best overall performance among all the competing methods.     

Improving the bandwidth-free inference methods by prewhitening

In this paper we consider inference of parameters in time series regression models. In the traditional inference approach, the heteroskedasticity and autocorrelation consistent (HAC) estimation is often involved to consistently estimate the asymptotic covariance matrix of regression parameter estimator. Since the bandwidth parameter in the HAC estimation is difficult to choose in practice, there has been a recent surge of interest in developing bandwidth-free inference methods. However, existing simulation studies show that these new methods suffer from severe size distortion in the presence of strong temporal dependence for a medium sample size. To remedy the problem, we propose to apply the prewhitening to the inconsistent long-run variance estimator in these methods to reduce the size distortion. The asymptotic distribution of the prewhitened Wald statistic is obtained and the general effectiveness of prewhitening is shown through simulations.     

Phase I Analysis of Multiple Linear Regression Profiles

This article considers the Phase I analysis of data when the quality of a process or product is characterized by a multiple linear regression model. This is usually referred to as the analysis of linear profiles in the statistical quality control literature. The literature includes several approaches for the analysis of simple linear regression profiles. Little work, however, has been done in the analysis of multiple linear regression profiles. This article proposes a new approach for the analysis of Phase I multiple linear regression profiles. Using this approach, regardless of the number of explanatory variables used to describe it, the profile response is monitored using only three parameters, an intercept, a slope, and a variance. Using simulation, the performance of the proposed method is compared to that of the existing methods for monitoring multiple linear profiles data in terms of the probability of a signal. The advantage of the proposed method over the existing methods is greatly improved detection of changes in the process parameters of linear profiles with high-dimensional space. The article also proposes useful diagnostic aids based on F-statistics to help in identifying the source of profile variation and the locations of out-of-control samples. Finally, the use of multiple linear profile methods is illustrated by a data set from a calibration application at National Aeronautics and Space Administration (NASA) Langley Research Center.     

Data cloning estimation of GARCH and COGARCH models

GARCH models include most of the stylized facts of financial time series and they have been largely used to analyse discrete financial time series. In the last years, continuous-time models based on discrete GARCH models have been also proposed to deal with non-equally spaced observations, as COGARCH model based on Lévy processes. In this paper, we propose to use the data cloning methodology in order to obtain estimators of GARCH and COGARCH model parameters. Data cloning methodology uses a Bayesian approach to obtain approximate maximum likelihood estimators avoiding numerically maximization of the pseudo-likelihood function. After a simulation study for both GARCH and COGARCH models using data cloning, we apply this technique to model the behaviour of some NASDAQ time series.     

基于均方根误差建模的多响应稳健参数设计

针对复杂产品质量设计阶段的静态多响应稳健参数设计中的模型参数不确定性问题,现有的多响应优化方法大都对响应的样本均值与样本方差采用双响应曲面法分别建模以考虑多响应的最优性与稳健性。文章在此基础上分析构建了均方根误差响应这一新的稳健性度量指标,提出了考虑模型参数不确定性的满意度函数方法,结合置信区间思想分析了模型参数不确定性对均方根误差响应的影响,并依据复杂产品质量设计阶段的实例进行分析研究,验证了该方法能够得到多响应系统在模型参数不确定性情况下更为稳健的全局最优解。     

Semiparametric Regression with Kernel Error Model

Abstract.  We propose and study a class of regression models, in which the mean function is specified parametrically as in the existing regression methods, but the residual distribution is modelled non-parametrically by a kernel estimator, without imposing any assumption on its distribution. This specification is different from the existing semiparametric regression models. The asymptotic properties of such likelihood and the maximum likelihood estimate (MLE) under this semiparametric model are studied. We show that under some regularity conditions, the MLE under this model is consistent (when compared with the possibly pseudo-consistency of the parameter estimation under the existing parametric regression model), is asymptotically normal with rate and efficient. The non-parametric pseudo-likelihood ratio has the Wilks property as the true likelihood ratio does. Simulated examples are presented to evaluate the accuracy of the proposed semiparametric MLE method.     

Likelihood ratio test-based chart for monitoring the process variability

This article proposes a new chart with the generalized likelihood ratio (GLR) test statistics for monitoring the process variance of a normally distributed process. The new chart can be easily designed and constructed and the computation results show that it provides quite a satisfactory performance, including the detection of the decrease in the variance and the individual observation at the sampling point which are very important in many practical applications. Average run length (ARL) comparisons between other procedures and the new chart are presented. The optimal parameters that can be used as a design aid in selecting specific parameter values based on the ARL are described. The application of our proposed method is illustrated by a real data example from chemical process control.     

Case-cohort analysis of clusters of recurrent events

The case-cohort sampling, first proposed in Prentice (Biometrika 73:1–11, 1986), is one of the most effective cohort designs for analysis of event occurrence, with the regression model being the typical Cox proportional hazards model. This paper extends to consider the case-cohort design for recurrent events with certain specific clustering feature, which is captured by a properly modified Cox-type self-exciting intensity model. We discuss the advantage of using this model and validate the pseudo-likelihood method. Simulation studies are presented in support of the theory. Application is illustrated with analysis of a bladder cancer data.     

Estimation of nonlinear simulation metamodels using control variates

The method of control variates has been intensively used for reducing the variance of estimated (linear) regression metamodels in simulation experiments. In contrast to previous studies, this article presents a procedure for applying multiple control variates when the objective is to estimate and validate a nonlinear regression metamodel for a single response, in terms of selected decision variables. This procedure includes robust statistical regression techniques for estimation and validation. Assuming joint normality of the response and controls, confidence intervals and hypothesis tests for the metamodel parameters are obtained. Finally, results for measuring the efficiency of the use of control variates are discussed.     

Quantile residual life regression based on semi-competing risks data

This paper investigates the quantile residual life regression based on semi-competing risk data. Because the terminal event time dependently censors the non-terminal event time, the inference on the non-terminal event time is not available without extra assumption. Therefore, we assume that the non-terminal event time and the terminal event time follow an Archimedean copula. Then, we apply the inverse probability weight technique to construct an estimating equation of quantile residual life regression coefficients. But, the estimating equation may not be continuous in coefficients. Thus, we apply the generalized solution approach to overcome this problem. Since the variance estimation of the proposed estimator is difficult to obtain, we use the bootstrap resampling method to estimate it. From simulations, it shows the performance of the proposed method is well. Finally, we analyze the Bone Marrow Transplant data for illustrations.     

A new linearized ridge Poisson estimator in the presence of multicollinearity

Poisson regression is a very commonly used technique for modeling the count data in applied sciences, in which the model parameters are usually estimated by the maximum likelihood method. However, the presence of multicollinearity inflates the variance of maximum likelihood (ML) estimator and the estimated parameters give unstable results. In this article, a new linearized ridge Poisson estimator is introduced to deal with the problem of multicollinearity. Based on the asymptotic properties of ML estimator, the bias, covariance and mean squared error of the proposed estimator are obtained and the optimal choice of shrinkage parameter is derived. The performance of the existing estimators and proposed estimator is evaluated through Monte Carlo simulations and two real data applications. The results clearly reveal that the proposed estimator outperforms the existing estimators in the mean squared error sense.KEYWORDS: Poisson regression, multicollinearity, ridge Poisson estimator, linearized ridge regression estimator, mean squared errorMathematics Subject Classifications: 62J07, 62F10