On estimating the dimensionality in discriminant analysis

In discriminant analysis, the dimension of the hyperplane which population mean vectors span is called the dimensionality. The procedures commonly used to estimate this dimension involve testing a sequence of dimensionality hypotheses as well as model fitting approaches based on (consistent) Akaike's method, (modified) Mallows' method and Schwarz's method. The marginal log-likelihood (MLL) method is developed and the asymptotic distribution of the dimensionality estimated by this method for normal populations is derived. Furthermore a modified marginal log-likelihood (MMLL) method is also considered. The MLL method is not consistent for large samples and two modified criteria are proposed which attain asymptotic consistency. Some comments are made with regard to the robustness of this method to departures from normality. The operating characteristics of the various methods proposed are examined and compared.     

Time series count data regression

The count data model studied in the paper extends the Poisson model by al-lowing for overdispersion and serial correlation. Alternative approaches to esti-mate nuisance parameters, required for the correction of the Poisson maximum likelihood covariance matrix estimator and for a quasi-likelihood estimator, are studied. The estimators are evaluated by finite sample Monte Carlo experi-mentation. It is found that the Poisson maximum likelihood estimator with corrected covariance matrix estimators provide reliable inferences for longer time series. Overdispersion test statistics are wellbehaved, while conventional portmanteau statistics for white noise have too large sizes. Two empirical illustrations are included.     

Estimation of gaussian mixtures with rotationally invariant covariance matrices

Homoscedastic and heteroscedastic Gaussian mixtures differ in the constraints placed on the covariance matrices of the mixture components. A new mixture, called herein a strophoscedastic mixture, is defined by a new constraint, This constraint requires the matrices to be identical under orthogonal trans¬formations, where different transformations are allowed for different matrices. It is shown that the M-step of the EM method for estimating the parameters of strophoscedastic mixtures from sample data is explicitly solvable using singular value decompositions. Consequently, the EM-based maximum likelihood estimation algorithm is as easily implemented for strophoscedastic mixtures as it is for homoscedastic and heteroscedastic mixtures. An example of a “noisy” Archimedian spiral is presented.     

Reverse Regression and Employment Discrimination

The importance of interval forecasts is reviewed. Several general approaches to calculating such forecasts are described and compared. They include the use of theoretical formulas based on a fitted probability model (with or without a correction for parameter uncertainty), various “approximate” formulas (which should be avoided), and empirically based, simulation, and resampling procedures. The latter are useful when theoretical formulas are not available or there are doubts about some model assumptions. The distinction between a forecasting method and a forecasting model is expounded. For large groups of series, a forecasting method may be chosen in a fairly ad hoc way. With appropriate checks, it may be possible to base interval forecasts on the model for which the method is optimal. It is certainly unsound to use a model for which the method is not optimal, but, strangely, this is sometimes done. Some general comments are made as to why prediction intervals tend to be too narrow in practice to encompass the required proportion of future observations. An example demonstrates the overriding importance of careful model specification. In particular, when data are “nearly nonstationary,” the difference between fitting a stationary and a nonstationary model is critical.     

Unobserved-Components Models for Seasonal Adjustment Filters

Time series models are presented, for which the seasonal-component estimates delivered by linear least squares signal extraction closely approximate those of the standard option of the widely-used Census X-11 program. Earlier work is extended by consideration of a broader class of models and by examination of asymmetric filters, in addition to the symmetric filter implicit in the adjustment of historical data. Various criteria that guide the specification of unobserved- components models are discussed, and a new preferred model is presented. Some nonstandard options in X-11 are considered in the Appendix.     

Tests for Cointegration Breakdown Over a Short Time Period

We derive a speculative trading model with endogenous informed trading that yields a conditionally heteroscedastic time series for trading volume and the squared price changes. We use half-hourly price-change and volume data for IBM during 1988 to test the model and estimate the structural parameters using the simulated method-of-moments estimation procedure. Although the model seems to do a reasonable job fitting the unconditional moments of the volume and the squared price change processes, it fares less well in fitting the relation between current trading volume and lags of trading volume and squared volume's (and its lag's) relation to squared price changes.     

The Design of Scoring Schemes for Surveys Using the Block Total Response Method

ABSTRACT

In this article we investigate the design of scoring schemes for surveys using the block total response method. This method was first proposed by Raghavarao and Federer (1979 Raghavarao , D. , Federer , W. T. ( 1979 ). Block total response as an alternative to the randomized response method in surveys . J. Roy. Statist. Soc. 41 ( 1 ): 40 – 45 . [CSA] [CROSSREF]  [Google Scholar]) to provide accurate estimates of the base rates of sensitive characteristics using balanced incomplete block designs. The scoring scheme used in Raghavarao and Federer (1979 Raghavarao , D. , Federer , W. T. ( 1979 ). Block total response as an alternative to the randomized response method in surveys . J. Roy. Statist. Soc. 41 ( 1 ): 40 – 45 . [CSA] [CROSSREF]  [Google Scholar]) did not guarantee anonymity of answers and so the possibility of improving on this basic scoring scheme is considered in this article.     

A Similarity Measure Based on Species Proportions

ABSTRACT

There are several indices for measuring the similarity of two populations, including the ratio of the number of shared species to the number of distinct species (Jaccard's index) and the conditional probability of observing a shared species (Smith et al., 1996 Smith , W. , Solow , A. R. , Preston , P. E. ( 1996 ). An estimator of species overlap using a modified beta-binomial model. Biometrics 52 : 1472 – 1477 . [CSA] [CROSSREF] [Crossref], [Web of Science ®] , [Google Scholar]). However, these indices only take into account the number of species and species proportions of shared species. In this article, we propose a new similarity index which includes the species proportions of both the shared and non shared species in each population, and also propose a Nonparametric Maximum Likelihood Estimator (NPMLE) for this index. Bootstrap and delta methods are used to evaluate the standard errors of the NPMLE. Based on a loss function, we also compare a class of nonparametric estimators for the proposed index in various situations.