An Adaptive Method of Variable Selection in Regression

An adaptive variable selection procedure is proposed which uses an adaptive test along with a stepwise procedure to select variables for a multiple regression model. We compared this adaptive stepwise procedure to methods that use Akaike's information criterion, Schwartz's information criterion, and Sawa's information criterion. The simulation studies demonstrated that the adaptive stepwise method is more effective than the traditional variable selection methods if the error distribution is not normally distributed. If the error distribution is known to be normally distributed, the variable selection method based on Sawa's information criteria appears to be superior to the other methods. Unless the error distribution is known to be normally distributed, the adaptive stepwise method is recommended.     

Entropy properties of record statistics

Record values can be viewed as order statistics from a sample whose size is determined by the values and the order of occurrence of observations. They are closely connected with the occurrence times of a corresponding non-homogenous Poisson process and reliability theory. In this paper, the information properties of record values are presented based on Shannon information. Several upper and lower bounds for the entropy of record values are obtained. It is shown that, the mutual information between record values is distribution free and is computable using the distribution of the record values of the sequence from the uniform distribution.     

Moment Information for Probability Distributions,Without Solving the Moment Problem. I: Where is the Mode?

How much information does a small number of moments carry about the unknown distribution function? Is it possible to explicitly obtain from these moments some useful information, e.g., about the support, the modality, the general shape, or the tails of a distribution, without going into a detailed numerical solution of the moment problem? In this paper a theoretical result of Johnson and Rogers is generalized to be valid for all moment problems and is exploited to demonstrate that a few moments are able to provide us with valuable information about the position of the mode of an unknown (unimodal) distribution.     

Information theoretic results for circular distributions

The broad class of generalized von Mises (GvM) circular distributions has certain optimal properties with respect to information theoretic quantities. It is shown that, under constraints on the trigonometric moments, and using the Kullback–Leibler information as the measure, the closest circular distribution to any other is of the GvM form. The lower bounds for the Kullback–Leibler information in this situation are also provided. The same problem is also considered using a modified version of the Kullback–Leibler information. Finally, series expansions are given for the entropy and the normalizing constants of the GvM distribution.     

Inferences on missing information under multiple imputation and two-stage multiple imputation

In the presence of missing values, researchers may be interested in the rates of missing information. The rates of missing information are (a) important for assessing how the missing information contributes to inferential uncertainty about, Q, the population quantity of interest, (b) are an important component in the decision of the number of imputations, and (c) can be used to test model uncertainty and model fitting. In this article I will derive the asymptotic distribution of the rates of missing information in two scenarios: the conventional multiple imputation (MI), and the two-stage MI. Numerically I will show that the proposed asymptotic distribution agrees with the simulated one. I will also suggest the number of imputations needed to obtain reliable missing information rate estimates for each method, based on the asymptotic distribution.     

Estimating Probability Distributions by Observing Betting Practices

A bookmaker takes bets on a two-horse race, attempting to minimize expected loss over all possible outcomes of the race. Profits are controlled by manipulation of customer' betting behavior; in order to do this, we need some information about the probability distribution which describes how the customers will bet. We examine what information initial customer' betting behavior provides about this probability distribution, and consider how to use this to estimate the probability distribution for remaining customers.     

Prediction for generalized linear models

Regression models are often used to make predictions. All the information needed is contained in the predictive distribution. However, this cannot be evaluated explicitly for most generalized linear models. We construct two approximations to this distribution and demonstrate their use on two sets of survival data, corresponding to the outcome of patients admitted to intensive care units and the survival times of leukaemia patients.Regression models are often used to make predictions. All the information needed is contained in the predictive distribution. However, this cannot be evaluated explicitly for most generalized linear models. We construct two approximations to this distribution and demonstrate their use on two sets of survival data, corresponding to the outcome of patients admitted to intensive care units and the survival times of leukaemia patients.Regression models are often used to make predictions. All the information needed is contained in the predictive distribution. However, this cannot be evaluated explicitly for most generalized linear models. We construct two approximations to this distribution and demonstrate their use on two sets of survival data, corresponding to the outcome of patients admitted to intensive care units and the survival times of leukaemia patients.Regression models are often used to make predictions. All the information needed is contained in the predictive distribution. However, this cannot be evaluated explicitly for most generalized linear models. We construct two approximations to this distribution and demonstrate their use on two sets of survival data, corresponding to the outcome of patients admitted to intensive care units and the survival times of leukaemia patients.     

A generally weighted moving average exceedance chart

Distribution-free control charts gained momentum in recent years as they are more efficient in detecting a shift when there is a lack of information regarding the underlying process distribution. However, a distribution-free control chart for monitoring the process location often requires information on the in-control process median. This is somewhat challenging because, in practice, any information on the location parameter might not be known in advance and estimation of the parameter is therefore required. In view of this, a time-weighted control chart, labelled as the Generally Weighted Moving Average (GWMA) exceedance (EX) chart (in short GWMA-EX chart), is proposed for detection of a shift in the unknown process location; this chart is based on exceedance statistic when there is no information available on the process distribution. An extensive performance analysis shows that the proposed GWMA-EX control chart is, in many cases, better than its contenders.     

Distribution estimation with auxiliary information for missing data

There is much literature on statistical inference for distribution under missing data, but surprisingly very little previous attention has been paid to missing data in the context of estimating distribution with auxiliary information. In this article, the auxiliary information with missing data is proposed. We use Zhou, Wan and Wang's method (2008) to mitigate the effects of missing data through a reformulation of the estimating equations, imputed through a semi-parametric procedure. Whence we can estimate distribution and the τth quantile of the distribution by taking auxiliary information into account. Asymptotic properties of the distribution estimator and corresponding sample quantile are derived and analyzed. The distribution estimators based on our method are found to significantly outperform the corresponding estimators without auxiliary information. Some simulation studies are conducted to illustrate the finite sample performance of the proposed estimators.     

Comparative analysis of population age-sex distribution from sample and complete enumeration: 1970 and 1980 census data

"This paper will attempt to analyze the degree of distortion introduced by sampling, if any, on the age and sex distribution of the population based on the information taken [in the 1970 census] from the sample households. Specifically, the 1970 age and sex distribution of the Philippine population based on the 5-percent sample households will be compared with that obtained from the information supplied by all households, i.e., the actual age and sex distribution of the Philippines....Findings...suggest a failure of the estimation procedures used in past censuses to correct errors attributable to sampling. In particular, the type of sampling error highlighted in this paper is the distortion introduced by sampling in age-sex distribution."     

Non-parametric Sampling Approximation via Voronoi Tessellations

In this article we propose a novel non-parametric sampling approach to estimate posterior distributions from parameters of interest. Starting from an initial sample over the parameter space, this method makes use of this initial information to form a geometrical structure known as Voronoi tessellation over the whole parameter space. This rough approximation to the posterior distribution provides a way to generate new points from the posterior distribution without any additional costly model evaluations. By using a traditional Markov Chain Monte Carlo (MCMC) over the non-parametric tessellation, the initial approximate distribution is refined sequentially. We applied this method to a couple of climate models to show that this hybrid scheme successfully approximates the posterior distribution of the model parameters.     

On the comparison of the Fisher information of the log-normal and generalized Rayleigh distributions

Surles and Padgett recently considered two-parameter Burr Type X distribution by introducing a scale parameter and called it the generalized Rayleigh distribution. It is observed that the generalized Rayleigh and log-normal distributions have many common properties and both distributions can be used quite effectively to analyze skewed data set. In this paper, we mainly compare the Fisher information matrices of the two distributions for complete and censored observations. Although, both distributions may provide similar data fit and are quite similar in nature in many aspects, the corresponding Fisher information matrices can be quite different. We compute the total information measures of the two distributions for different parameter ranges and also compare the loss of information due to censoring. Real data analysis has been performed for illustrative purposes.     

Some Characterizations Based on Entropy of Order Statistics and Record Values

In the literature of information theory, Shannon entropy plays an important role and in the context of reliability theory, order statistics and record values are used for statistical modeling. The aim of this article is characterizing the parent distributions based on Shannon entropy of order statistics and record values. It is shown that the equality of the Shannon information in order statistics or record values can determine uniquely the parent distribution. The exponential distribution is characterized through maximizing Shannon entropy of record values under some constraints. The results are useful in the modeling problems.     

On the choice of the prior distribution in hypergeometric sampling

Information in a statistical procedure arising from sources other than sampling is called prior information, and its incorporation into the procedure forms the basis of the Bayesian approach to statistics. Under hypergeometric sampling, methodology is developed which quantifies the amount of information provided by the sample data relative to that provided by the prior distribution and allows for a ranking of prior distributions with respect to conservativeness, where conservatism refers to restraint of extraneous information embedded in any prior distribution. The most conservative prior distribution from a specified class (each member of which carries the available prior information) is that prior distribution within the class over which the likelihood function has the greatest average domination. Four different families of prior distributions are developed by considering a Bayesian approach to the formation of lots. The most conservative prior distribution from each of the four families of prior distributions is determined and compared for the situation when no prior information is available. The results of the comparison advocate the use of the Polya (beta-binomial) prior distribution in hypergeometric sampling.     

A note on the asymptotic distribution of the maximum likelihood estimator for the scalar skew-normal distribution

We consider likelihood based inference for the parameter of a skew-normal distribution. One of the problems shown by this model is the singularity of the Fisher information matrix when skewness is absent. We derive the rate of convergence to the asymptotic distribution of the maximum likelihood estimator and study an alternative parameterization which overcomes problems related to the singularity of the information matrix.     

Bayesian analysis of a multiple-recapture model

In the Bayesian analysis of a multiple-recapture census, different diffuse prior distributions can lead to markedly different inferences about the population size N. Through consideration of the Fisher information matrix it is shown that the number of captures in each sample typically provides little information about N. This suggests that if there is no prior information about capture probabilities, then knowledge of just the sample sizes and not the number of recaptures should leave the distribution of Nunchanged. A prior model that has this property is identified and the posterior distribution is examined. In particular, asymptotic estimates of the posterior mean and variance are derived. Differences between Bayesian and classical point and interval estimators are illustrated through examples.     

A bayesian analysis of interrelated bernoulli processes with an application for clinical trials

Prior information regarding the interrelation of two Bernoulli processes may justify a clinical trial designed to corroborate this information. Antelman (1973) has studied the Dirichlet-beta which permits the expression of the prior knowledge of such interrelation. However, use of this prior distribution leads to complicated and intractable analyses. Alternately, such prior information regarding the interrelation of the processes may be adequately summarized by a simple Dirichlet distribution. Procedures for testing hypotheses regarding a priori interrelations of the success probabilities of the processes are given. Exact expressions for the posterior probabi1ities of these hypotheses are shown to be approximately equal to weighted p-values or 1ikelihood ratios.     

ESTIMATING DISTRIBUTION FUNCTIONS FROM SURVEY DATA WITH LIMITED BENCHMARK INFORMATION

The model-based approach to estimation of finite population distribution functions introduced in Chambers & Dunstan (1986) is extended to the case where only summary information is available for the auxiliary size variable. Monte Carlo results indicate that this ‘limited information’ extension is almost as efficient as the ‘full information’ method proposed in the above reference. These results also indicate that the model-based confidence intervals generated by either of these methods have superior coverage properties to more conventional design-based confidence intervals.     

Fisher Information for Two Gamma Frailty Bivariate Weibull Models

The asymptotic properties of frailty models for multivariate survival data are not well understood. To study this aspect, the Fisher information is derived in the standard bivariate gamma frailty model, where the survival distribution is of Weibull form conditional on the frailty. For comparison, the Fisher information is also derived in the bivariate gamma frailty model, where the marginal distribution is of Weibull form.     

Bayesian Estimation for Deformed Modified Power Series Distribution

In this article, posterior distribution, posterior moments, and predictive distribution for the modified power series distributions deformed at any of a support point under linex and generalized entropy loss function are derived. It is assumed that the prior information can be summarized by a uniform, Beta, two-sided power, Gamma, or generalized Pareto distributions. The obtained results are demonstrated on the generalized Poisson and the generalized negative binomial distribution deformed at a given point.