Type I error rates from likelihood‐based repeated measures analyses of incomplete longitudinal data

The last observation carried forward (LOCF) approach is commonly utilized to handle missing values in the primary analysis of clinical trials. However, recent evidence suggests that likelihood‐based analyses developed under the missing at random (MAR) framework are sensible alternatives. The objective of this study was to assess the Type I error rates from a likelihood‐based MAR approach – mixed‐model repeated measures (MMRM) – compared with LOCF when estimating treatment contrasts for mean change from baseline to endpoint (Δ). Data emulating neuropsychiatric clinical trials were simulated in a 4 × 4 factorial arrangement of scenarios, using four patterns of mean changes over time and four strategies for deleting data to generate subject dropout via an MAR mechanism. In data with no dropout, estimates of Δ and SE_Δ from MMRM and LOCF were identical. In data with dropout, the Type I error rates (averaged across all scenarios) for MMRM and LOCF were 5.49% and 16.76%, respectively. In 11 of the 16 scenarios, the Type I error rate from MMRM was at least 1.00% closer to the expected rate of 5.00% than the corresponding rate from LOCF. In no scenario did LOCF yield a Type I error rate that was at least 1.00% closer to the expected rate than the corresponding rate from MMRM. The average estimate of SE_Δ from MMRM was greater in data with dropout than in complete data, whereas the average estimate of SE_Δ from LOCF was smaller in data with dropout than in complete data, suggesting that standard errors from MMRM better reflected the uncertainty in the data. The results from this investigation support those from previous studies, which found that MMRM provided reasonable control of Type I error even in the presence of MNAR missingness. No universally best approach to analysis of longitudinal data exists. However, likelihood‐based MAR approaches have been shown to perform well in a variety of situations and are a sensible alternative to the LOCF approach. MNAR methods can be used within a sensitivity analysis framework to test the potential presence and impact of MNAR data, thereby assessing robustness of results from an MAR method. Copyright © 2004 John Wiley & Sons, Ltd.     

Detecting multiple change points in piecewise constant hazard functions

The National Cancer Institute (NCI) suggests a sudden reduction in prostate cancer mortality rates, likely due to highly successful treatments and screening methods for early diagnosis. We are interested in understanding the impact of medical breakthroughs, treatments, or interventions, on the survival experience for a population. For this purpose, estimating the underlying hazard function, with possible time change points, would be of substantial interest, as it will provide a general picture of the survival trend and when this trend is disrupted. Increasing attention has been given to testing the assumption of a constant failure rate against a failure rate that changes at a single point in time. We expand the set of alternatives to allow for the consideration of multiple change-points, and propose a model selection algorithm using sequential testing for the piecewise constant hazard model. These methods are data driven and allow us to estimate not only the number of change points in the hazard function but where those changes occur. Such an analysis allows for better understanding of how changing medical practice affects the survival experience for a patient population. We test for change points in prostate cancer mortality rates using the NCI Surveillance, Epidemiology, and End Results dataset.     

Type I and Type II error rates in the last observation carried forward method under informative dropout

Dropout is a persistent problem for a longitudinal study. We exhibit the shortcomings of the last observation carried forward method. It produces biased estimates of change in an outcome from baseline to study endpoint under informative dropout. We developed a theoretical quantification of the effect of such bias on type I and type II error rates. We present results for a setup where a subject either completes the study or drops out during one particular interval, and also under the setup in which subjects could drop out at any time during the study. The type I error rate steadily increases when time to dropout decreases or the common sample size increases. The inflation in type I error rate can be substantially high when reasons for dropout in the two groups differ; when there is a large difference in dropout rates between the control and treatment groups and when the common sample size is large; even when dropout subjects have one or two fewer observations than the completers. Similar results are also observed for type II error rates. A study can have very low power when early recovered patients in the treatment group and worsening patients in the control group drop out even near the end of the study.     

Different methods for handling incomplete longitudinal binary outcome due to missing at random dropout

This paper compares the performance of weighted generalized estimating equations (WGEEs), multiple imputation based on generalized estimating equations (MI-GEEs) and generalized linear mixed models (GLMMs) for analyzing incomplete longitudinal binary data when the underlying study is subject to dropout. The paper aims to explore the performance of the above methods in terms of handling dropouts that are missing at random (MAR). The methods are compared on simulated data. The longitudinal binary data are generated from a logistic regression model, under different sample sizes. The incomplete data are created for three different dropout rates. The methods are evaluated in terms of bias, precision and mean square error in case where data are subject to MAR dropout. In conclusion, across the simulations performed, the MI-GEE method performed better in both small and large sample sizes. Evidently, this should not be seen as formal and definitive proof, but adds to the body of knowledge about the methods’ relative performance. In addition, the methods are compared using data from a randomized clinical trial.     

Statistical analysis of right-censored failure-time data with partially specified hazard rates

This paper discusses the analysis of right-censored failure-time data in which the failure rate may have different forms in different time intervals. Such data occur naturally, for example, in demography studies and leukemia research, and a number of methods for the analysis have been proposed in the literature. However, most methods are purely parametric or nonparametric. Matthews and Farewell (1982), for example, discussed this problem and proposed a method for testing a constant failure rate against a failure rate involving a change point. To estimate an absolute limit on the attainable human life span, Zelterman (1992) discussed a hazard function that has different parametric forms over different time intervals. We consider a different situation in which the hazard function may follow a parametric form before a change point and is completely unknown after the change point. To test the existence of the change point, a modified maximal-censored-likelihood-ratio test is proposed and its asymptotic properties are studied. A bootstrap method is described for finding critical values of the proposed test. Simulation results indicate that the test performs well.     

Bias in the last observation carried forward method under informative dropout

Subject dropout is an inevitable problem in longitudinal studies. It makes the analysis challenging when the main interest is the change in outcome from baseline to endpoint of study. The last observation carried forward (LOCF) method is a very common approach for handling this problem. It assumes that the last measured outcome is frozen in time after the point of dropout, an unrealistic assumption given any time trends. Though existence and direction of the bias can sometimes be anticipated, the more important statistical question involves the actual magnitude of the bias and this requires computation. This paper provides explicit expressions for the exact bias in the LOCF estimates of mean change and its variance when the longitudinal data follow a linear mixed-effects model with linear time trajectories. General dropout patterns are considered that may depend on treatment group, subject-specific trajectories and follow different time to dropout distributions. In our case studies, the magnitude of bias for mean change estimators linearly increases as time to dropout decreases. The bias depends heavily on the dropout interval. The variance term is always underestimated.     

Multiple imputation compared with restricted pseudo‐likelihood and generalized estimating equations for analysis of binary repeated measures in clinical studies

Non‐likelihood‐based methods for repeated measures analysis of binary data in clinical trials can result in biased estimates of treatment effects and associated standard errors when the dropout process is not completely at random. We tested the utility of a multiple imputation approach in reducing these biases. Simulations were used to compare performance of multiple imputation with generalized estimating equations and restricted pseudo‐likelihood in five representative clinical trial profiles for estimating (a) overall treatment effects and (b) treatment differences at the last scheduled visit. In clinical trials with moderate to high (40–60%) dropout rates with dropouts missing at random, multiple imputation led to less biased and more precise estimates of treatment differences for binary outcomes based on underlying continuous scores. Copyright © 2005 John Wiley & Sons, Ltd.     

Semiparametric Sequential Testing for Multiple Change Points in Piecewise Constant Hazard Functions with Long-Term Survivors

In this article, we consider parameter estimation in the hazard rate with multiple change points in the presence of long-term survivors. We combine two methods: maximum likelihood based and martingale based, to estimate the change points in the hazard rate for right censored survival data that accounts for long-term survivors. A simulation study is carried out to compare the performance of estimators. The method is applied to analyze two real datasets.     

AN APPROACH FOR JOINTLY MODELING MULTIVARIATE LONGITUDINAL MEASUREMENTS AND DISCRETE TIME-TO-EVENT DATA

In many medical studies, patients are followed longitudinally and interest is on assessing the relationship between longitudinal measurements and time to an event. Recently, various authors have proposed joint modeling approaches for longitudinal and time-to-event data for a single longitudinal variable. These joint modeling approaches become intractable with even a few longitudinal variables. In this paper we propose a regression calibration approach for jointly modeling multiple longitudinal measurements and discrete time-to-event data. Ideally, a two-stage modeling approach could be applied in which the multiple longitudinal measurements are modeled in the first stage and the longitudinal model is related to the time-to-event data in the second stage. Biased parameter estimation due to informative dropout makes this direct two-stage modeling approach problematic. We propose a regression calibration approach which appropriately accounts for informative dropout. We approximate the conditional distribution of the multiple longitudinal measurements given the event time by modeling all pairwise combinations of the longitudinal measurements using a bivariate linear mixed model which conditions on the event time. Complete data are then simulated based on estimates from these pairwise conditional models, and regression calibration is used to estimate the relationship between longitudinal data and time-to-event data using the complete data. We show that this approach performs well in estimating the relationship between multivariate longitudinal measurements and the time-to-event data and in estimating the parameters of the multiple longitudinal process subject to informative dropout. We illustrate this methodology with simulations and with an analysis of primary biliary cirrhosis (PBC) data.     

Sequential combining in discriminant analysis

In practice, it often happens that we have a number of base methods of classification. We are not able to clearly determine which method is optimal in the sense of the smallest error rate. Then we have a combined method that allows us to consolidate information from multiple sources in a better classifier. I propose a different approach, a sequential approach. Sequentiality is understood here in the sense of adding posterior probabilities to the original data set and so created data are used during classification process. We combine posterior probabilities obtained from base classifiers using all combining methods. Finally, we combine these probabilities using a mean combining method. To the original data set we add obtained posterior probabilities as additional features. In each step we change our additional probabilities to achieve the minimum error rate for base methods. Experimental results on different data sets demonstrate that the method is efficient and that this approach outperforms base methods providing a reduction in the mean classification error rate.     

STATISTICAL INFERENCE OF TRANSMISSION FIDELITY OF DNA METHYLATION PATTERNS OVER SOMATIC CELL DIVISIONS IN MAMMALS

We develop Bayesian inference methods for a recently-emerging type of epigenetic data to study the transmission fidelity of DNA methylation patterns over cell divisions. The data consist of parent-daughter double-stranded DNA methylation patterns with each pattern coming from a single cell and represented as an unordered pair of binary strings. The data are technically difficult and time-consuming to collect, putting a premium on an efficient inference method. Our aim is to estimate rates for the maintenance and de novo methylation events that gave rise to the observed patterns, while accounting for measurement error. We model data at multiple sites jointly, thus using whole-strand information, and considerably reduce confounding between parameters. We also adopt a hierarchical structure that allows for variation in rates across sites without an explosion in the effective number of parameters. Our context-specific priors capture the expected stationarity, or near-stationarity, of the stochastic process that generated the data analyzed here. This expected stationarity is shown to greatly increase the precision of the estimation. Applying our model to a data set collected at the human FMR1 locus, we find that measurement errors, generally ignored in similar studies, occur at a non-trivial rate (inappropriate bisulfite conversion error: 1.6% with 80% CI: 0.9-2.3%). Accounting for these errors has a substantial impact on estimates of key biological parameters. The estimated average failure of maintenance rate and daughter de novo rate decline from 0.04 to 0.024 and from 0.14 to 0.07, respectively, when errors are accounted for. Our results also provide evidence that de novo events may occur on both parent and daughter strands: the median parent and daughter de novo rates are 0.08 (80% CI: 0.04-0.13) and 0.07 (80% CI: 0.04-0.11), respectively.     

Marginal and Conditional Distribution Estimation from Double‐sampled Semi‐competing Risks Data

Informative dropout is a vexing problem for any biomedical study. Most existing statistical methods attempt to correct estimation bias related to this phenomenon by specifying unverifiable assumptions about the dropout mechanism. We consider a cohort study in Africa that uses an outreach programme to ascertain the vital status for dropout subjects. These data can be used to identify a number of relevant distributions. However, as only a subset of dropout subjects were followed, vital status ascertainment was incomplete. We use semi‐competing risk methods as our analysis framework to address this specific case where the terminal event is incompletely ascertained and consider various procedures for estimating the marginal distribution of dropout and the marginal and conditional distributions of survival. We also consider model selection and estimation efficiency in our setting. Performance of the proposed methods is demonstrated via simulations, asymptotic study and analysis of the study data.     

Calculating power for the comparison of dependent κ-coefficients

Summary. In the psychosocial and medical sciences, some studies are designed to assess the agreement between different raters and/or different instruments. Often the same sample will be used to compare the agreement between two or more assessment methods for simplicity and to take advantage of the positive correlation of the ratings. Although sample size calculations have become an important element in the design of research projects, such methods for agreement studies are scarce. We adapt the generalized estimating equations approach for modelling dependent κ -statistics to estimate the sample size that is required for dependent agreement studies. We calculate the power based on a Wald test for the equality of two dependent κ -statistics. The Wald test statistic has a non-central χ ²-distribution with non-centrality parameter that can be estimated with minimal assumptions. The method proposed is useful for agreement studies with two raters and two instruments, and is easily extendable to multiple raters and multiple instruments. Furthermore, the method proposed allows for rater bias. Power calculations for binary ratings under various scenarios are presented. Analyses of two biomedical studies are used for illustration.     

Estimating individual-level interaction effects in multilevel models: a Monte Carlo simulation study with application*

Moderated multiple regression provides a useful framework for understanding moderator variables. These variables can also be examined within multilevel datasets, although the literature is not clear on the best way to assess data for significant moderating effects, particularly within a multilevel modeling framework. This study explores potential ways to test moderation at the individual level (level one) within a 2-level multilevel modeling framework, with varying effect sizes, cluster sizes, and numbers of clusters. The study examines five potential methods for testing interaction effects: the Wald test, F-test, likelihood ratio test, Bayesian information criterion (BIC), and Akaike information criterion (AIC). For each method, the simulation study examines Type I error rates and power. Following the simulation study, an applied study uses real data to assess interaction effects using the same five methods. Results indicate that the Wald test, F-test, and likelihood ratio test all perform similarly in terms of Type I error rates and power. Type I error rates for the AIC are more liberal, and for the BIC typically more conservative. A four-step procedure for applied researchers interested in examining interaction effects in multi-level models is provided.     

Nonparametric tests for right-censored data with biased sampling

Testing the equality of two survival distributions can be difficult in a prevalent cohort study when non random sampling of subjects is involved. Due to the biased sampling scheme, independent censoring assumption is often violated. Although the issues about biased inference caused by length-biased sampling have been widely recognized in statistical, epidemiological and economical literature, there is no satisfactory solution for efficient two-sample testing. We propose an asymptotic most efficient nonparametric test by properly adjusting for length-biased sampling. The test statistic is derived from a full likelihood function, and can be generalized from the two-sample test to a k-sample test. The asymptotic properties of the test statistic under the null hypothesis are derived using its asymptotic independent and identically distributed representation. We conduct extensive Monte Carlo simulations to evaluate the performance of the proposed test statistics and compare them with the conditional test and the standard logrank test for different biased sampling schemes and right-censoring mechanisms. For length-biased data, empirical studies demonstrated that the proposed test is substantially more powerful than the existing methods. For general left-truncated data, the proposed test is robust, still maintains accurate control of type I error rate, and is also more powerful than the existing methods, if the truncation patterns and right-censoring patterns are the same between the groups. We illustrate the methods using two real data examples.     

Ecological inference techniques: an empirical evaluation using data describing gender and voter turnout at New Zealand elections, 1893–1919

Summary.  The difference, if any, between men's and women's voting patterns is of particular interest to historians of gender and politics. For elections that were held before the introduction of opinion surveying in the 1940s, little data are available with which to estimate such differences. We apply six methods for ecological inference to estimate men's and women's voting rates in New Zealand (NZ), 1893–1919. NZ is an interesting case-study, since it was the first self-governing country where women could vote. Furthermore, NZ officials recorded the voting rates of men and women at elections, making it possible to compare estimates produced by methods for ecological inference with known true values, thus testing the efficacy of different methods for ecological inference for this data set. We find that the most popular methods for ecological inference, namely Goodman's ecological regression and King's parametric method, give poor estimates, as does the much debated neighbourhood method. However, King's non-parametric method, Chambers and Steel's semiparametric method and the Steel, Beh and Chambers homogeneous approach all gave good estimates that were close to the known values, with the homogeneous approach performing best overall. The success of these methods in this example suggests that ecological inference may be a viable option when investigating gender and voting. Moreover, researchers using ecological inference in other fields may do well to consider a range of statistical methods. This work is a significant NZ contribution to historical politics and the first quantitative contribution, in the area of NZ gender and politics.     

Identifying Structural Models of Committee Decisions With Heterogeneous Tastes and Ideological Bias

In practice, members of a committee often make different recommendations despite a common goal and shared sources of information. We study the nonparametric identification and estimation of a structural model, where such discrepancies are rationalized by the members’ unobserved types, which consist of ideological bias while weighing different sources of information, and tastes for multiple objectives announced in the policy target. We consider models with and without strategic incentives for members to make recommendations that conform to the final committee decision. We show that pure-strategy Bayesian Nash equilibria exist in both cases, and that the variation in common information recorded in the data helps us to recover the distribution of private types from the members’ choices. Building on the identification result, we estimate a structural model of interest rate decisions by the Monetary Policy Committee (MPC) at the Bank of England. We find some evidence that the external committee members are less affected by strategic incentives for conformity in their recommendations than the internal members. We also find that the difference in ideological bias between external and internal members is statistically insignificant. Supplementary materials for this article are available online.     

Partial central subspace and sliced average variance estimation

Sliced average variance estimation is one of many methods for estimating the central subspace. It was shown to be more comprehensive than sliced inverse regression in the sense that it consistently estimates the central subspace under mild conditions while slice inverse regression may estimate only a proper subset of the central subspace. In this paper we extend this method to regressions with qualitative predictors. We also provide tests of dimension and a marginal coordinate hypothesis test. We apply the method to a data set concerning lakes infested by Eurasian Watermilfoil, and compare this new method to the partial inverse regression estimator.     

A method for testing interaction in unreplicated two-way tables: using all pairwise interaction contrasts

Several methods exist for testing interaction in unreplicated two-way layouts. Some are based on specifying a functional form for the interaction term and perform well provided that the functional form is appropriate. Other methods do not require such a functional form to be specified but only test for the presence of non-additivity and do not provide a suitable estimate of error variance for a non-additive model. This paper presents a method for testing for interaction in unreplicated two-way tables that is based on testing all pairwise interaction contrasts. This method (i) is easy to implement, (ii) does not assume a functional form for the interaction term, (iii) can find a sub-table of data which may be free from interaction and to base the estimate of unknown error variance, and (iv) can be used for incomplete two-way layouts. The proposed method is illustrated using examples and its power is investigated via simulation studies. Simulation results show that the proposed method is competitive with existing methods for testing for interaction in unreplicated two-way layouts.     

Multiple comparisons in two-factor testing of psychophysical thresholds by the delta test method

We propose an algorithm of multiple comparisons in two-factor testing of psychophysical thresholds. We use logistic regression combined with guessing rate and adopt the step-down procedure with Ryan–Einot–Gabriel–Welsch (REGW) significance levels for multiple test. We test hypotheses of main effects and interaction in two-factor problem by using the delta test statistics.