Using quantile regression for duration analysis

Summary Quantile regression methods are emerging as a popular technique in econometrics and biometrics for exploring the distribution of duration data. This paper discusses quantile regression for duration analysis allowing for a flexible specification of the functional relationship and of the error distribution. Censored quantile regression addresses the issue of right censoring of the response variable which is common in duration analysis. We compare quantile regression to standard duration models. Quantile regression does not impose a proportional effect of the covariates on the hazard over the duration time. However, the method cannot take account of time-varying covariates and it has not been extended so far to allow for unobserved heterogeneity and competing risks. We also discuss how hazard rates can be estimated using quantile regression methods. This paper benefitted from the helpful comments by an anonymous referee. Due to space constraints, we had to omit the details of the empirical application. These can be found in the long version of this paper, Fitzenberger and Wilke (2005). We gratefully acknowledge financial support by the German Research Foundation (DFG) through the research project ‘Microeconometric modelling of unemployment durations under consideration of the macroeconomic situation’. Thanks are due to Xuan Zhang for excellent research assistance. All errors are our sole responsibility.     

P-splines regression smoothing and difference type of penalty

P-splines regression provides a flexible smoothing tool. In this paper we consider difference type penalties in a context of nonparametric generalized linear models, and investigate the impact of the order of the differencing operator. Minimizing Akaike’s information criterion we search for a possible best data-driven value of the differencing order. Theoretical derivations are established for the normal model and provide insights into a possible ‘optimal’ choice of the differencing order and its interrelation with other parameters. Applications of the selection procedure to non-normal models, such as Poisson models, are given. Simulation studies investigate the performance of the selection procedure and we illustrate its use on real data examples.     

Unemployment duration and the length of entitlement periods for unemployment benefits: do the IAB employment subsample and the German Socio-Economic Panel yield the same results?*

Summary: We compare information on the length of unemployment spells contained in the IAB employment subsample (IABS) and in the German Socio-Economic Panel (GSOEP). Due to the lack of information on registered unemployment in the IABS, we use two proxies of unemployment in the IABS as introduced by Fitzenberger/Wilke (2004). The first proxy comprises all periods of nonemployment after an employment spell which contain at least one period with unemployment compensation transfers. The second proxy includes all episodes between two employment spells during which an individual continuously received unemployment benefits. Estimation of standard duration models indicates that conclusions drawn from the IABS and the GSOEP differ in many cases. While the GSOEP suggests that the hazard rate has a maximum at about 12 months of unemployment, the IABS results suggest that this maximum is at about 20 months. Contrary to our GSOEP results and contrary to many results based on the GSOEP found in the literature, we find a statistically significant association between longer maximum entitlement periods of unemployment benefits (‘Arbeitslosengeld’) and longer unemployment durations for men in the IABS. The results for women do not show such clear patterns. The large sample size of the IABS also allows to trace out statistically significant effects of characteristics such as regional and industry indicators, which is generally not possible in the relatively small GSOEP. * Acknowledgements: We would like to thank the editors of this special issue, Joachim M?ller and Bernd Fitzenberger, two anonymous referees, the participants of the ‘Statistische Woche 2004’ in Frankfurt (in particular Reinhard Hujer, Olaf Hübler and Gerd Ronning), seminar participants at the ZEW Mannheim (especially Fran?ois Laisney and Alexander Spermann) and Jennifer Hunt for their many helpful comments and suggestions. All remaining errors are our own. Financial support of the Deutsche Forschungsgemeinschaft (DFG) through the research project ‘Microeconometric modelling of unemployment durations under consideration of the macroeconomic situation’ is gratefully acknowledged. The data used in this paper were made available by the Institute for Employment Research (IAB) at the Federal Labour Office of Germany, Nürnberg, and the German Socio Economic Panel Study (GSOEP) at the German Institute for Economic Research (DIW), Berlin.     

Perturbation by multiplicative noise and the Simulation Extrapolation method

While most of the literature on measurement error focuses on additive measurement error, we consider in this paper the multiplicative case. We apply the Simulation Extrapolation method (SIMEX)—a procedure which was originally proposed by Cook and Stefanski (J. Am. Stat. Assoc. 89:1314–1328, 1994) in order to correct the bias due to additive measurement error—to the case where data are perturbed by multiplicative noise and present several approaches to account for multiplicative noise in the SIMEX procedure. Furthermore, we analyze how well these approaches reduce the bias caused by multiplicative perturbation. Using a binary probit model, we produce Monte Carlo evidence on how the reduction of data quality can be minimized. For helpful comments, we would like to thank Helmut Küchenhoff, Winfried Pohlmeier, and Gerd Ronning. Sandra Nolte gratefully acknowledges financial support by the DFG. Elena Biewen and Martin Rosemann gratefully acknowledge the financial support by the Federal Ministry of Education and Research (BMBF). The usual disclaimer applies.     

Estimation of linear models with anonymised panel data

The paper analyses the biasing effect of anonymising micro data by multiplicative stochastic noise on the within estimation of a linear panel model. In short panels, additional bias results from serially correlated regressors. Results in this paper are related to the project “Firms’ Panel Data and Factual Anonymisation,” which is financed by Federal Ministry of Education and Research. We would like to thank the anonymous referees for helpful comments.     

Econometric analysis of high frequency data

Summary Owing to enormous advances in data acquisition and processing technology the study of high (or ultra) frequency data has become an important area of econometrics. At least three avenues of econometric methods have been followed to analyze high frequency financial data: Models in tick time ignoring the time dimension of sampling, duration models specifying the time span between transactions and, finally, fixed time interval techniques. Starting from the strong assumption that quotes are irregularly generated from an underlying exogeneous arrival process, fixed interval models promise feasibility of familiar time series techniques. Moreover, fixed interval analysis is a natural means to investigate multivariate dynamics. In particular, models of price discovery are implemented in this venue of high frequency econometrics. Recently, a sound statistical theory of ‘realized volatility’ has been developed. In this framework high frequency log price changes are seen as a means to observe volatility at some lower frequency.     

Fuzzy measures of the incidence of relative poverty and deprivation: a multi-dimensional perspective

In this paper we present a methodology for the study of multi-dimensional aspects of poverty and deprivation. The conventional poor/non-poor dichotomy is replaced by defining poverty as a matter of degree, determined by the place of the individual in the income distribution. The fuzzy poverty measure proposed is in fact also expressible in terms of the generalised Gini measure. The same methodology facilitates the inclusion of other dimensions of deprivation into the analysis: by appropriately weighting indicators of deprivation to reflect their dispersion and correlation, we can construct measures of non-monetary deprivation in its various dimensions. These indicators illuminate the extent to which purely monetary indicators are insufficient in themselves in capturing the prevalence of deprivation. An important contribution of the paper is to identify rules for the aggregation of fuzzy sets appropriate for the study of poverty and deprivation. In particular, we define a ‘composite’ fuzzy set operator which takes into account whether the sets being aggregated are of a ‘similar’ or a ‘dissimilar’ type. These rules allow us to meaningfully combine income and the diverse non-income deprivation indices at the micro-level and construct what we have termed ‘intensive’ and ‘extensive’ indicators of deprivation. We note that mathematically the same approach can be carried over to the study of persistence of poverty and deprivation over time.     

A ‘nondecimated’ lifting transform

Classical nondecimated wavelet transforms are attractive for many applications. When the data comes from complex or irregular designs, the use of second generation wavelets in nonparametric regression has proved superior to that of classical wavelets. However, the construction of a nondecimated second generation wavelet transform is not obvious. In this paper we propose a new ‘nondecimated’ lifting transform, based on the lifting algorithm which removes one coefficient at a time, and explore its behavior. Our approach also allows for embedding adaptivity in the transform, i.e. wavelet functions can be constructed such that their smoothness adjusts to the local properties of the signal. We address the problem of nonparametric regression and propose an (averaged) estimator obtained by using our nondecimated lifting technique teamed with empirical Bayes shrinkage. Simulations show that our proposed method has higher performance than competing techniques able to work on irregular data. Our construction also opens avenues for generating a ‘best’ representation, which we shall explore.     

Modeling individual migraine severity with autoregressive ordered probit models

This paper considers the problem of modeling migraine severity assessments and their dependence on weather and time characteristics. We take on the viewpoint of a patient who is interested in an individual migraine management strategy. Since factors influencing migraine can differ between patients in number and magnitude, we show how a patient’s headache calendar reporting the severity measurements on an ordinal scale can be used to determine the dominating factors for this special patient. One also has to account for dependencies among the measurements. For this the autoregressive ordinal probit (AOP) model of Müller and Czado (J Comput Graph Stat 14: 320–338, 2005) is utilized and fitted to a single patient’s migraine data by a grouped move multigrid Monte Carlo (GM-MGMC) Gibbs sampler. Initially, covariates are selected using proportional odds models. Model fit and model comparison are discussed. A comparison with proportional odds specifications shows that the AOP models are preferred.     

Regression Estimation Using Multivariate Failure Time Data and a Common Baseline Hazard Function Model

Recent ‘marginal’ methods for the regression analysis of multivariate failure time data have mostly assumed Cox (1972)model hazard functions in which the members of the cluster have distinct baseline hazard functions. In some important applications, including sibling family studies in genetic epidemiology and group randomized intervention trials, a common baseline hazard assumption is more natural. Here we consider a weighted partial likelihood score equation for the estimation of regression parameters under a common baseline hazard model, and provide corresponding asymptotic distribution theory. An extensive series of simulation studies is used to examine the adequacy of the asymptotic distributional approximations, and especially the efficiency gain due to weighting, as a function of strength of dependency within cluster, and cluster size. This revised version was published online in July 2006 with corrections to the Cover Date.     

A sickle transition-rate model with starting threshold

Billari (2001) introduced a new type of single-spell parametric transition-rate model: transition-rate models with a starting threshold. In such models, the transition-rate function is composed of two additive terms. The first term is a constant that holds for any given duration; the second is a ‘traditional’ transition-rate function with the threshold as its time origin, and it is added after a certain threshold point. The possibility of allowing for the presence of long-term survivors in the social process has not yet been dealt with, and it is of specific interest in several domains of application. In this paper, we develop the specific case of the sickle model. We discuss its features, its implementation as a starting threshold model, and the estimation of its parameters. The sickle model with starting threshold is then applied to the union formation of Italian men and women, using the Fertility and Family Survey data.     

Flexible competing risks regression modeling and goodness-of-fit

In this paper we consider different approaches for estimation and assessment of covariate effects for the cumulative incidence curve in the competing risks model. The classic approach is to model all cause-specific hazards and then estimate the cumulative incidence curve based on these cause-specific hazards. Another recent approach is to directly model the cumulative incidence by a proportional model (Fine and Gray, J Am Stat Assoc 94:496–509, 1999), and then obtain direct estimates of how covariates influences the cumulative incidence curve. We consider a simple and flexible class of regression models that is easy to fit and contains the Fine–Gray model as a special case. One advantage of this approach is that our regression modeling allows for non-proportional hazards. This leads to a new simple goodness-of-fit procedure for the proportional subdistribution hazards assumption that is very easy to use. The test is constructive in the sense that it shows exactly where non-proportionality is present. We illustrate our methods to a bone marrow transplant data from the Center for International Blood and Marrow Transplant Research (CIBMTR). Through this data example we demonstrate the use of the flexible regression models to analyze competing risks data when non-proportionality is present in the data.     

Spectral estimation for locally stationary time series with missing observations

Time series arising in practice often have an inherently irregular sampling structure or missing values, that can arise for example due to a faulty measuring device or complex time-dependent nature. Spectral decomposition of time series is a traditionally useful tool for data variability analysis. However, existing methods for spectral estimation often assume a regularly-sampled time series, or require modifications to cope with irregular or ‘gappy’ data. Additionally, many techniques also assume that the time series are stationary, which in the majority of cases is demonstrably not appropriate. This article addresses the topic of spectral estimation of a non-stationary time series sampled with missing data. The time series is modelled as a locally stationary wavelet process in the sense introduced by Nason et al. (J. R. Stat. Soc. B 62(2):271–292, 2000) and its realization is assumed to feature missing observations. Our work proposes an estimator (the periodogram) for the process wavelet spectrum, which copes with the missing data whilst relaxing the strong assumption of stationarity. At the centre of our construction are second generation wavelets built by means of the lifting scheme (Sweldens, Wavelet Applications in Signal and Image Processing III, Proc. SPIE, vol. 2569, pp. 68–79, 1995), designed to cope with irregular data. We investigate the theoretical properties of our proposed periodogram, and show that it can be smoothed to produce a bias-corrected spectral estimate by adopting a penalized least squares criterion. We demonstrate our method with real data and simulated examples.     

Almost Consistent Estimation of Panel Probit Models with “Small” Fixed Effects

We propose four different GMM estimators that allow almost consistent estimation of the structural parameters of panel probit models with fixed effects for the case of small Tand large N. The moments used are derived for each period from a first order approximation of the mean of the dependent variable conditional on explanatory variables and on the fixed effect. The estimators differ w.r.t. the choice of instruments and whether they use trimming to reduce the bias or not. In a Monte Carlo study, we compare these estimators with pooled probit and conditional logit estimators for different data generating processes. The results show that the proposed estimators outperform these competitors in several situations.     

Probabilistic relabelling strategies for the label switching problem in Bayesian mixture models

The label switching problem is caused by the likelihood of a Bayesian mixture model being invariant to permutations of the labels. The permutation can change multiple times between Markov Chain Monte Carlo (MCMC) iterations making it difficult to infer component-specific parameters of the model. Various so-called ‘relabelling’ strategies exist with the goal to ‘undo’ the label switches that have occurred to enable estimation of functions that depend on component-specific parameters. Existing deterministic relabelling algorithms rely upon specifying a loss function, and relabelling by minimising its posterior expected loss. In this paper we develop probabilistic approaches to relabelling that allow for estimation and incorporation of the uncertainty in the relabelling process. Variants of the probabilistic relabelling algorithm are introduced and compared to existing deterministic relabelling algorithms. We demonstrate that the idea of probabilistic relabelling can be expressed in a rigorous framework based on the EM algorithm.     

The rules of conditional inference: Is there a universal definition of nonformation?

Summary We consider the ideas of sufficiency and ancillarity for parametric models with nuisance parameters, and more generally Barndorff-Nielsen's notion of nonformation. The original four definitions of non-formation, namelyB-,S-,G- andM-nonformation, each cover different types of models. We stress the interpretation of nonformation in terms of the idea of perfect fit. This leads to a new definition of nonformation, calledI-nonformation, which is well suited for inference in exponential families. We also consider Rémon's concept ofL-sufficiency, and a recent extension toL-nonformation, due to Barndorff-Nielsen, which unifies and extendsB-,S- andG- nonformation. We study the relations between these six definitions, and show that they are all special cases ofM-nonformation. All animals are equal, but some animals are more equal than others. From ‘Animal Farm’, by G. Orwell (1945).     

Feasible estimation in generalized structured models

This article introduces a feasible estimation method for a large class of semi and nonparametric models. We present the family of generalized structured models which we wish to estimate. After highlighting the main idea of the theoretical smooth backfitting estimators, we introduce a general estimation procedure. We consider modifications and practical issues, and discuss inference, cross validation, and asymptotic theory applying the theoretical framework of Mammen and Nielsen (Biometrika 90: 551–566, 2003). An extensive simulation study shows excellent performance of our method. Furthermore, real data applications from environmetrics and biometrics demonstrate its usefulness.     

Adjusting for time-varying confounding in the subdistribution analysis of a competing risk

Despite decades of research in the medical literature, assessment of the attributable mortality due to nosocomial infections in the intensive care unit (ICU) remains controversial, with different studies describing effect estimates ranging from being neutral to extremely risk increasing. Interpretation of study results is further hindered by inappropriate adjustment (a) for censoring of the survival time by discharge from the ICU, and (b) for time-dependent confounders on the causal path from infection to mortality. In previous work (Vansteelandt et al. Biostatistics 10:46–59), we have accommodated this through inverse probability of treatment and censoring weighting. Because censoring due to discharge from the ICU is so intimately connected with a patient’s health condition, the ensuing inverse weighting analyses suffer from influential weights and rely heavily on the assumption that one has measured all common risk factors of ICU discharge and mortality. In this paper, we consider ICU discharge as a competing risk in the sense that we aim to infer the risk of ‘ICU mortality’ over time that would be observed if nosocomial infections could be prevented for the entire study population. For this purpose we develop marginal structural subdistribution hazard models with accompanying estimation methods. In contrast to subdistribution hazard models with time-varying covariates, the proposed approach (a) can accommodate high-dimensional confounders, (b) avoids regression adjustment for post-infection measurements and thereby so-called collider-stratification bias, and (c) results in a well-defined model for the cumulative incidence function. The methods are used to quantify the causal effect of nosocomial pneumonia on ICU mortality using data from the National Surveillance Study of Nosocomial Infections in ICU’s (Belgium).     

Contingency tables with prescribed marginals

For an adjustment of contingency tables to prescribed marginal frequencies Deming and Stephan (1940) minimize a Chi-square expression. Asymptotically equivalently, Ireland and Kullback (1968) minimize a Leibler-Kullback divergence, where the probabilistical arguments for both methods remain vague. Here we deduce a probabilistical model based on observed contingency tables. It shows that the two above methods and the maximum likelihood approach in Smith (1947) yield asymptotically the ‘most probable’ adjustment under prescribed marginal frequencies. The fundamental hypothesis of statistical mechanics relates observations to ‘most probable’ realizations. ‘Most probable’ is going to be used in the sense of so-called large deviations. The proposed adjustment has a significant product form and will be generalized to contingency tables with infinitely many cells.