SIMPLE LM TESTS FOR THE UNBALANCED NESTED ERROR COMPONENT REGRESSION MODEL

This paper derives several Lagrange Multiplier tests for the unbalanced nested error component model. Economic data with a natural nested grouping include firms grouped by industry; or students grouped by schools. The LM tests derived include the joint test for both effects as well as the test for one effect conditional on the presence of the other. The paper also derives the standardized versions of these tests, their asymptotic locally mean most powerful version as well as their robust to local misspecification version. Monte Carlo experiments are conducted to study the performance of these LM tests.     

Nonparametric kernel density estimation for general grouped data

Interval-grouped data are defined, in general, when the event of interest cannot be directly observed and it is only known to have been occurred within an interval. In this framework, a nonparametric kernel density estimator is proposed and studied. The approach is based on the classical Parzen–Rosenblatt estimator and on the generalisation of the binned kernel density estimator. The asymptotic bias and variance of the proposed estimator are derived under usual assumptions, and the effect of using non-equally spaced grouped data is analysed. Additionally, a plug-in bandwidth selector is proposed. Through a comprehensive simulation study, the behaviour of both the estimator and the plug-in bandwidth selector considering different scenarios of data grouping is shown. An application to real data confirms the simulation results, revealing the good performance of the estimator whenever data are not heavily grouped.     

A Poisson-multinomial mixture approach to grouped and right-censored counts

Although count data are often collected in social, psychological, and epidemiological surveys in grouped and right-censored categories, there is a lack of statistical methods simultaneously taking both grouping and right-censoring into account. In this research, we propose a new generalized Poisson-multinomial mixture approach to model grouped and right-censored (GRC) count data. Based on a mixed Poisson-multinomial process for conceptualizing grouped and right-censored count data, we prove that the new maximum-likelihood estimator (MLE-GRC) is consistent and asymptotically normally distributed for both Poisson and zero-inflated Poisson models. The use of the MLE-GRC, implemented in an R function, is illustrated by both statistical simulation and empirical examples. This research provides a tool for epidemiologists to estimate incidence from grouped and right-censored count data and lays a foundation for regression analyses of such data structure.     

Estimation of parameters in logistic and log-logistic distribution with grouped data

In many situations, instead of a complete sample, data are available only in grouped form. For example, grouped failure time data occur in studies in which subjects are monitored periodically to determine whether failure has occurred in the predetermined intervals. Here the model under consideration is the log-logistic distribution. This paper demonstrates the existence and uniqueness of the MLEs of the parameters of the logistic distribution under mild conditions with grouped data. The times with the maximum failure rate and the mode of the p.d.f. of the log-logistic distribution are also estimated based on the MLEs. The methodology is further studied with simulations and exemplified with a data set with artificially introduced grouping from a locomotive life test study.     

Checking the Grouped Data Version of the Cox Model for Interval-grouped Survival Data

Abstract.  Epidemiology research often entails the analysis of failure times subject to grouping. In large cohorts interval grouping also offers a feasible choice of data reduction to actually facilitate an analysis of the data. Based on an underlying Cox proportional hazards model for the exact failure times one may deduce a grouped data version of this model which may then be used to analyse the data. The model bears a lot of resemblance to a generalized linear model, yet due to the nature of data one also needs to incorporate censoring. In the case of non-trivial censoring this precludes model checking procedures based on ordinary residuals as calculation of these requires knowledge of the censoring distribution. In this paper, we represent interval grouped data in a dynamical way using a counting process approach. This enables us to identify martingale residuals which can be computed without knowledge of the censoring distribution. We use these residuals to construct graphical as well as numerical model checking procedures. An example from epidemiology is provided.     

Optimum group limits for estimation in scaled log-logistic distribution from a grouped data

The well known logistic distribution is considered. A transformation of the logistic variate in terms of exponential function results in a new distribution called log-logistic distribution suggested by Balakrishnanet al (1987). Estimation of its scale parameter from a grouped data is presented. Optimal group limits in the case of equispaced as well as unequispaced groupings so as to have a maximum asymptotic relative efficiency are worked out. The grouping correction in the case of equispaced grouped data with a mid point type estimator is also suggested. The results are expalined by an example.     

GROUPING CORRECTIONS FOR SEQUENTIAL TESTS

When testing sequentially with grouped data, we may or may not adjust the test statistic to allow for the grouping. If the test statistic is not adjusted, the whole power curve will be altered by the grouping, while if the statistic is adjusted, the power curve is altered, but the probability levels or errors of the first and second kind, being distribution-free, are unchanged. It is shown, even for very coarse grouping of the data, that the power curve is only slightly affected by using the unadjusted statistic for tests on the mean or the variance of a normal distribution. For certain other distributions, however, in particular the negative exponential distribution, this procedure breaks down completely if the group width exceeds a certain value.     

On the grouped selection and model complexity of the adaptive elastic net

Lasso proved to be an extremely successful technique for simultaneous estimation and variable selection. However lasso has two major drawbacks. First, it does not enforce any grouping effect and secondly in some situation lasso solutions are inconsistent for variable selection. To overcome this inconsistency adaptive lasso is proposed where adaptive weights are used for penalizing different coefficients. Recently a doubly regularized technique namely elastic net is proposed which encourages grouping effect i.e. either selection or omission of the correlated variables together. However elastic net is also inconsistent. In this paper we study adaptive elastic net which does not have this drawback. In this article we specially focus on the grouped selection property of adaptive elastic net along with its model selection complexity. We also shed some light on the bias-variance tradeoff of different regularization methods including adaptive elastic net. An efficient algorithm was proposed in the line of LARS-EN, which is then illustrated with simulated as well as real life data examples.     

Bootstrap likelihood ratio test for Weibull mixture models fitted to grouped data

Abstract

Weibull mixture models are widely used in a variety of fields for modeling phenomena caused by heterogeneous sources. We focus on circumstances in which original observations are not available, and instead the data comes in the form of a grouping of the original observations. We illustrate EM algorithm for fitting Weibull mixture models for grouped data and propose a bootstrap likelihood ratio test (LRT) for determining the number of subpopulations in a mixture model. The effectiveness of the LRT methods are investigated via simulation. We illustrate the utility of these methods by applying them to two grouped data applications.     

GLOBAL LOG-CONCAVITY OF THE LIKELIHOOD IN MODELS WITH GROUPED DISCRETE DATA

The paper considers the property of global log-concavity of the likelihood function in discrete data models in which the data are observed in ‘grouped’ form, meaning that for some observations, while the actual value is unknown, the realisation of the discrete random variable is known to fall within a certain range of values. A typical likelihood contribution in this type of model is a sum of probabilities over a range of realisations. An important issue is whether the property of log-concavity in the ungrouped case carries over to the grouped counterpart; the paper finds, by way of a simple but relevant counter-example, that this is not always the case. However, in two cases of practical interest, namely the Poisson and geometric models, the property of log-concavity is preserved under grouping.     

Grouped poisson regression models: theory and an application to public house visit frequency

In this paper we extend the Poisson regression model to deal with the situation in which the event count is observed le in “grouped” form, By this we mean that for some observations, all that is known about the count is that it falls within a certain range of integers, and the actual value is unknown, A typical likelihood contribution for this extended model is the sum of a set of consecutive Poisson probabilities, The log-likelihood function is derived for a general grouping rule, using a logarithmic link for the Poisson mean, This log-likelihood function is shown to be globally concave. The model is applied to grouped count data on the frequency of trips to pubs made over a one-week period by a sample of Norfolk young persons.     

Median split, k-group split, and optimality in continuous populations

Continuous populations are grouped in many social, economic, medical, or technical fields of research. However, by grouping them, a lot of information provided by the continuous population is lost. Especially the median split, which is still adopted by many researchers, and its generalization to an equiprobable k-group split lead to a high efficiency loss. Here, this loss of information is investigated by analytical and numerical analyses for some typical symmetric and skew population distributions often found in applications. Various distribution parameters, numbers of groups, and split methods are taken from theoretical considerations and real data sets. Losses sometimes in excess of 50% can be reduced by optimal grouping.     

On the History of the Correction for Grouping, 1873–1922

The correction for grouping is a sum of two terms, the first depending on the length of the grouping interval, the second being a periodic function of the position. Thiele (1873) studied the second term, but missed the first. Sheppard (1898) studied the first term, but missed the second. Bruns (1906) derived the first term as the aperiodic term of a Fourier series and the second as the sum of the periodic terms. He found the correction to the coefficients of the Gram–Charlier series and proved that the second term is negligible for a grouped normal distribution with at least eight groups. Independently, Fisher (1922) used the same method to derive the correction to the moments. For the normal distribution with a grouping interval less than the standard deviation Fisher proved that the second term is negligible compared with the first and with the standard error of the first four moments. Moreover, he proved that the estimates of the mean and the standard deviation obtained by the method of moments for a grouped sample with Sheppard's corrections have nearly the same variances as the maximum likelihood estimates, thus providing a new and compelling reason for using Sheppard's corrections.     

Bayesian assessment of goodness of fit against nonparametric alternatives

The classical chi‐square test of goodness of fit compares the hypothesis that data arise from some parametric family of distributions, against the nonparametric alternative that they arise from some other distribution. However, the chi‐square test requires continuous data to be grouped into arbitrary categories. Furthermore, as the test is based upon an approximation, it can only be used if there are sufficient data. In practice, these requirements are often wasteful of information and overly restrictive. The authors explore the use of the fractional Bayes factor to obtain a Bayesian alternative to the chi‐square test when no specific prior information is available. They consider the extent to which their methodology can handle small data sets and continuous data without arbitrary grouping.     

Grouping Variable Selection by Weight Fused Elastic Net for Multi-Collinear Data

In this article, we consider the problem of variable selection and estimation with the strongly correlated multi-collinear data by using grouping variable selection techniques. A new grouping variable selection method, called weight-fused elastic net(WFEN), is proposed to deal with the high dimensional collinear data. The proposed model, combined two different grouping effect mechanisms induced by the elastic net and weight-fused LASSO, respectively, can be easily unified in the frame of LASSO and computed efficiently. The performance with the simulation and real data sets shows that our method is competitive with other related methods, especially when the data present high multi-collinearity.     

Quantile estimation from grouped data: the cell midpoint

Data are often collected in histogram form, especially in the context of computer simulation. While requiring less memory and computation than saving all observations, the grouping of observations in the histogram cells complicates statistical estimation of parameters of interest. In this paper the mean and variance of the cell midpoint estimator of the p^th quantile are analyzed in terms of distribution, cell width, and sample size. Three idiosyncrasies of using cell midpoints to estimate quantiles are illustrated. The results tend to run counter to previously published results for grouped data.     

Grouping Tests for Misspecification: An Application to Housing Demand

We consider several grouping tests for regression misspecification, with reference to housing-demand function estimation. We compare three existing test procedures, demonstrate modifications necessary in most applications, and propose a fourth test to distinguish between two categories of potential specification error. The test procedures are evaluated in artificial simulations of alternative errors. Finally, we apply the tests to FHA home purchase data. We reject the hypothesis that household and grouped regressions differ only by sampling error or random mismeasurement of household income or price. Our results have implications for choices among test procedures and interpretations of previous housing-demand analysis.     

A fiducial approach to multiple comparisons

Comparing treatment means from populations that follow independent normal distributions is a common statistical problem. Many frequentist solutions exist to test for significant differences amongst the treatment means. A different approach would be to determine how likely it is that particular means are grouped as equal. We developed a fiducial framework for this situation. Our method provides fiducial probabilities that any number of means are equal based on the data and the assumed normal distributions. This methodology was developed when there is constant and non-constant variance across populations. Simulations suggest that our method selects the correct grouping of means at a relatively high rate for small sample sizes and asymptotic calculations demonstrate good properties. Additionally, we have demonstrated the flexibility in the methods ability to calculate the fiducial probability for any number of equal means. This was done by analyzing a simulated data set and a data set measuring the nitrogen levels of red clover plants that were inoculated with different treatments.     

Comparing a survey and a conjoint study: the future vision of water intermediaries

This paper compares and contrasts two methods of obtaining opinions using questionnaires. As the name suggests, a conjoint study makes it possible to consider several attributes jointly. Conjoint analysis is a statistical method to analyse preferences. However, conjoint analysis requires a certain amount of effort by the respondent. The alternative is ordinary survey questions, answered one at a time. Survey questions are easier to grasp mentally, but they do not challenge the respondent to prioritize. This investigation has utilized both methods, survey and conjoint, making it possible to compare them on real data. Attribute importance, attribute correlations, case clustering and attribute grouping are evaluated by both methods. Correspondence between how the two methods measure the attribute in question is also given. Overall, both methods yield the same picture concerning the relative importance of the attributes. Taken one attribute at a time, the correspondence between the methods varies from good to no correspondence. Considering all attributes together by cluster analysis of the cases, the conjoint and survey data yield different cluster structures. The attributes are grouped by factor analysis, and there is reasonable correspondence. The data originate from the EU project 'New Intermediary services and the transformation of urban water supply and wastewater disposal systems in Europe'.     

Grouped data exponentially weighted moving average control charts

In the manufacture of metal fasteners in a progressive die operation, and other industrial situations, important quality dimensions cannot be measured on a continuous scale, and manufactured parts are classified into groups by using a step gauge. This paper proposes a version of exponentially weighted moving average (EWMA) control charts that are applicable to monitoring the grouped data for process shifts. The run length properties of this new grouped data EWMA chart are compared with similar results previously obtained for EWMA charts for variables data and with those for cumulative sum (CUSUM) schemes based on grouped data. Grouped data EWMA charts are shown to be nearly as efficient as variables-based EWMA charts and are thus an attractive alternative when the collection of variables data is not feasible. In addition, grouped data EWMA charts are less affected by the discreteness that is inherent in grouped data than are grouped data CUSUM charts. In the metal fasteners application, grouped data EWMA charts were simple to implement and allowed the rapid detection of undesirable process shifts.