Discriminant analysis under the common principal components model

For two or more populations of which the covariance matrices have a common set of eigenvectors, but different sets of eigenvalues, the common principal components (CPC) model is appropriate. Pepler et al. (2015 Pepler, P. T., Uys, D. W. and Nel, D. G. (2015). Regularised covariance matrix estimation under the common principal components model. Communications in Statistics: Simulation and Computation. (In press). [Google Scholar]) proposed a regularized CPC covariance matrix estimator and showed that this estimator outperforms the unbiased and pooled estimators in situations, where the CPC model is applicable. This article extends their work to the context of discriminant analysis for two groups, by plugging the regularized CPC estimator into the ordinary quadratic discriminant function. Monte Carlo simulation results show that CPC discriminant analysis offers significant improvements in misclassification error rates in certain situations, and at worst performs similar to ordinary quadratic and linear discriminant analysis. Based on these results, CPC discriminant analysis is recommended for situations, where the sample size is small compared to the number of variables, in particular for cases where there is uncertainty about the population covariance matrix structures.     

A combinatorial study of two-periodic random walks

Two-periodic random walks have up-steps and down-steps of one unit as usual, but the probability of an up-step is α after an even number of steps and β = 1 ? α after an odd number of steps, and reversed for down-steps. This concept was studied by Böhm and Hornik^{[2 Böhm, W.; Hornik, K. On two-periodic random walks with boundaries. Stoch. Models 2010, 26, 165–194.[Taylor &; Francis Online], [Web of Science ®] , [Google Scholar]]}. We complement this analysis by using methods from (analytic) combinatorics. By using two steps at once, we can reduce the analysis to the study of Motzkin paths, with up-steps, down-steps, and level-steps. Using a proper substitution, we get the generating functions of interest in an explicit and neat form. The parameters that are discussed here are the (one-sided) maximum (already studied by Böhm and Hornik^{[2 Böhm, W.; Hornik, K. On two-periodic random walks with boundaries. Stoch. Models 2010, 26, 165–194.[Taylor &; Francis Online], [Web of Science ®] , [Google Scholar]]}) and the two-sided maximum. For the asymptotic evaluation of the average value of the two-sided maximum after n random steps, more sophisticated methods from complex analysis (Mellin transform, singularity analysis) are required. The approach to transfer the analysis to Motzkin paths is, of course, not restricted to the two parameters under consideration.     

Robustness of the Affine Equivariant Scatter Estimator Based on the Spatial Rank Covariance Matrix

Visuri et al. (2000 Visuri, S., Koivunen, V., Oja, H. (2000). Sign and rank covariance matrices. J. Stat. Plann. Inference 91:557–575.[Crossref], [Web of Science ®] , [Google Scholar]) proposed a technique for robust covariance matrix estimation based on different notions of multivariate sign and rank. Among them, the spatial rank based covariance matrix estimator that utilizes a robust scale estimator is especially appealing due to its high robustness, computational ease, and good efficiency. Also, it is orthogonally equivariant under any distribution and affinely equivariant under elliptically symmetric distributions. In this paper, we study robustness properties of the estimator with respective to two measures: breakdown point and influence function. More specifically, the upper bound of the finite sample breakdown point can be achieved by a proper choice of univariate robust scale estimator. The influence functions for eigenvalues and eigenvectors of the estimator are derived. They are found to be bounded under some assumptions. Moreover, finite sample efficiency comparisons to popular robust MCD, M, and S estimators are reported.     

A new kind of stochastic restricted biased estimator for logistic regression model

In the logistic regression model, the variance of the maximum likelihood estimator is inflated and unstable when the multicollinearity exists in the data. There are several methods available in literature to overcome this problem. We propose a new stochastic restricted biased estimator. We study the statistical properties of the proposed estimator and compare its performance with some existing estimators in the sense of scalar mean squared criterion. An example and a simulation study are provided to illustrate the performance of the proposed estimator.KEYWORDS: Logistic regression, maximum likelihood estimator, mean squared error matrix, ridge regression, simulation study, stochastic restricted estimatorMathematics Subject Classifications: Primary 62J05, Secondary 62J07     

Matrix geometric approach for random walks: Stability condition and equilibrium distribution

ABSTRACT

In this paper, we analyze a sub-class of two-dimensional homogeneous nearest neighbor (simple) random walk restricted on the lattice using the matrix geometric approach. In particular, we first present an alternative approach for the calculation of the stability condition, extending the result of Neuts drift conditions^[30 Neuts, M.F., Matrix-geometric Solutions in Stochastic Models: An Algorithmic Approach; The Johns Hopkins University Press: Baltimore, 1981. [Google Scholar]^] and connecting it with the result of Fayolle et al. which is based on Lyapunov functions.^[13 Fayolle, G.; Iasnogorodski, R.; Malyshev, V., Random Walks in the Quarter Plane; Springer-Verlag: New York, 1999. [Google Scholar]^] Furthermore, we consider the sub-class of random walks with equilibrium distributions given as series of product forms and, for this class of random walks, we calculate the eigenvalues and the corresponding eigenvectors of the infinite matrix R appearing in the matrix geometric approach. This result is obtained by connecting and extending three existing approaches available for such an analysis: the matrix geometric approach, the compensation approach and the boundary value problem method. In this paper, we also present the spectral properties of the infinite matrix R.     

Monitoring means and covariances of multivariate non linear time series with heavy tails

In this paper, the focus is on sequential analysis of multivariate financial time series with heavy tails. The mean vector and the covariance matrix of multivariate non linear models are simultaneously monitored by modifying conventional control charts to identify structural changes in the data. The considered target process is a constant conditional correlation model (cf. Bollerslev, 1990 Bollerslev, T. (1990). Modeling the coherence in short-run nominal exchange rates: A multivariate generalized ARCH model. Rev. Econ. Stat. 72:498–505.[Crossref], [Web of Science ®] , [Google Scholar]), an extended constant conditional correlation model (cf. He and Teräsvirta, 2004 He, C., Teräsvirta, T. (2004). An extended constant conditional correlation GARCH model and its fourth-moment structure. Economet. Theory 20:904–926.[Crossref], [Web of Science ®] , [Google Scholar]), a dynamic conditional correlation model (cf. Engle, 2002 Engle, R.F. (2002). Dynamic conditional correlation: A simple class of multivariate GARCH models. J. Bus. Econ. Stat. 20(3):339–350.[Taylor &; Francis Online], [Web of Science ®] , [Google Scholar]), or a generalized dynamic conditional correlation model (cf. Capiello et al., 2006 Capiello, L., Engle, R., Sheppard, K. (2006). Asymmetric correlations in the dynamics of global equity and bond returns. J. Financial Economet. 4(4):537–572.[Crossref] , [Google Scholar]). For statistical surveillance we use control charts based on residuals. Further, the procedures are constructed for t-distribution. The detection speed of these charts is compared via Monte Carlo simulation. In the empirical study, the procedure with the best performance is applied to log-returns of the stock market indices FTSE and CAC.     

Empirical Likelihood Method for Censored Median Regression Models

Based on the semiparametric median regression analysis for the right-censored data developed by Ying et al. (1995 Ying , Z. , Jung , S. H. , Wei , L. J. ( 1995 ). Survival analysis with median regression models . J. Amer. Statist. Assoc. 90 : 178 – 184 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]), an empirical likelihood based inferential procedure for the regression coefficients is proposed. The limiting distribution of the proposed log-empirical likelihood ratio test statistic follows a chi-squared distribution, which corresponds to the standard asymptotic results of the empirical likelihood method. The inference about the subsets of the entire regression coefficients vector is discussed. The proposed method is illustrated by some simulation studies.     

A review of tests for exponentiality with Monte Carlo comparisons

In this paper, 91 different tests for exponentiality are reviewed. Some of the tests are universally consistent while others are against some special classes of life distributions. Power performances of 40 of these different tests for exponentiality of datasets are compared through extensive Monte Carlo simulations. The comparisons are conducted for different sample sizes of 10, 25, 50 and 100 for different groups of distributions according to the shape of their hazard functions at 5 percent level of significance. Also, the techniques are applied to two real-world datasets and a measure of power is employed for the comparison of the tests. The results show that some tests which are very good under one group of alternative distributions are not so under another group. Also, some tests maintained relatively high power over all the groups of alternative distributions studied while some others maintained poor power performances over all the groups of alternative distributions. Again, the result obtained from real-world datasets agree completely with those of the simulation studies.KEYWORDS: Classes of life distributions, empirical power of a test, exponentiality, goodness-of-fit test, Monte Carlo simulationSubject Classifications: 62E10, 62E20, 62F03     

Selection of Binary Variables and Classification by Boosting

We adopt boosting for classification and selection of high-dimensional binary variables for which classical methods based on normality and non singular sample dispersion are inapplicable. Boosting seems particularly well suited for binary variables. We present three methods of which two combine boosting with the relatively classical variable selection methods developed in Wilbur et al. (2002 Wilbur , J. D. , Ghosh , J. K. , Nakatsu , C. H. , Brouder , S. M. , Doerge , R. W. ( 2002 ). Variable selection in high-dimensional multivariate binary data with application to the analysis of microbial community DNA fingerprints . Biometrics 58 : 378 – 386 . [Google Scholar]). Our primary interest is variable selection in classification with small misclassification error being used as validation of proposed method for variable selection. Two of the new methods perform uniformly better than Wilbur et al. (2002 Wilbur , J. D. , Ghosh , J. K. , Nakatsu , C. H. , Brouder , S. M. , Doerge , R. W. ( 2002 ). Variable selection in high-dimensional multivariate binary data with application to the analysis of microbial community DNA fingerprints . Biometrics 58 : 378 – 386 . [Google Scholar]) in one set of simulated and three real life examples.     

A Consistent Method for the Selection of Relevant Instruments

Abstract

In many applications, a researcher must select an instrument vector from a candidate set of instruments. If the ultimate objective is to perform inference about the unknown parameters using conventional asymptotic theory, then we argue that it is desirable for the chosen instrument vector to satisfy four conditions which we refer to as orthogonality, identification, efficiency, and non‐redundancy. It is impossible to verify a priori which elements of the candidate set satisfy these conditions; this can only be done using the data. However, once the data are used in this fashion it is important that the selection process does not contaminate the limiting distribution of the parameter estimator. We refer to this requirement as the inference condition. In a recent paper, Andrews [[Andrews, D. W. K. (1999)] Andrews, D. W.K. 1999. Consistent moment selection procedures for generalized method of moments estimation. Econometrica, 67: 543–564. [Crossref], [Web of Science ®] , [Google Scholar]. Consistent moment selection procedures for generalized method of moments estimation. Econometrica67:543–564] has proposed a method of moment selection based on an information criterion involving the overidentifying restrictions test. This method can be shown to select an instrument vector which satisfies the orthogonality condition with probability one in the limit. In this paper, we consider the problem of instrument selection based on a combination of the efficiency and non‐redundancy conditions which we refer to as the relevance condition. It is shown that, within a particular class of models, certain canonical correlations form the natural metric for relevancy, and this leads us to propose a canonical correlations information criterion (CCIC) for instrument selection. We establish conditions under which our method satisfies the inference condition. We also consider the properties of an instrument selection method based on the sequential application of [Andrews, D. W. K. (1999)] Andrews, D. W.K. 1999. Consistent moment selection procedures for generalized method of moments estimation. Econometrica, 67: 543–564. [Crossref], [Web of Science ®] , [Google Scholar]. Consistent moment selection procedures for generalized method of moments estimation. Econometrica67:543–564 method and CCIC.     

Multivariate singular spectrum analysis for forecasting revisions to real-time data

Real-time data on national accounts statistics typically undergo an extensive revision process, leading to multiple vintages on the same generic variable. The time between the publication of the initial and final data is a lengthy one and raises the question of how to model and forecast the final vintage of data – an issue that dates from seminal articles by Mankiw et al. [51 Mankiw, N. G., Runkle, M. and Shapiro, M. D. 1984. Are preliminary announcements of the money stock rational forecasts?. J. Monetary Econ., 14: 15–27. [Crossref], [Web of Science ®] [Google Scholar]], Mankiw and Shapiro [52 Mankiw, N. G. and Shapiro, M. D. 1986. News or noise? An analysis of GNP revisions. Surv. Curr. Bus. May, : 20–25.  [Google Scholar]] and Nordhaus [57 Nordhaus, W. D. 1987. Forecasting efficiency: Concepts and applications. Rev. Econ. Stat., 4: 667–674.  [Google Scholar]]. To solve this problem, we develop the non-parametric method of multivariate singular spectrum analysis (MSSA) for multi-vintage data. MSSA is much more flexible than the standard methods of modelling that involve at least one of the restrictive assumptions of linearity, normality and stationarity. The benefits are illustrated with data on the UK index of industrial production: neither the preliminary vintages nor the competing models are as accurate as the forecasts using MSSA.     

On Tests Based on Sample Quasi Ranges for Ordered Alternative (Scale Case of Exponential Distribution)

The analysis of categorical response data through the multinomial model is very frequent in many statistical, econometric, and biometric applications. However, one of the main problems is the precise estimation of the model parameters when the number of observations is very low. We propose a new Bayesian estimation approach where the prior distribution is constructed through the transformation of the multivariate beta of Olkin and Liu (2003 Olkin , I. , Liu , R. ( 2003 ). A bivariate beta distribution . Stat. Probab. Lett. 62 : 407 – 412 .[Crossref], [Web of Science ®] , [Google Scholar]). Moreover, the application of the zero-variance principle allows us to estimate moments in Monte Carlo simulations with a dramatic reduction of their variances. We show the advantages of our approach through applications to some toy examples, where we get efficient parameter estimates.     

The Exact General Formulas for the Moments of a Ridge Regression Estimator when the Regression Error Terms Follow a Multivariate t Distribution

Huang (1999 Huang , J. C. ( 1999 ). Improving the estimation precision for a selected parameter in multiple regression analysis: an algebraic approach . Econ. Lett. 62 : 261 – 264 .[Crossref], [Web of Science ®] , [Google Scholar]) proposed a feasible ridge regression (FRR) estimator to estimate a specific regression coefficient. Assuming that the error terms follow a normal distribution, Huang (1999 Huang , J. C. ( 1999 ). Improving the estimation precision for a selected parameter in multiple regression analysis: an algebraic approach . Econ. Lett. 62 : 261 – 264 .[Crossref], [Web of Science ®] , [Google Scholar]) examined the small sample properties of the FRR estimator. In this article, assuming that the error terms follow a multivariate t distribution, we derive an exact general formula for the moments of the FRR estimator to estimate a specific regression coefficient. Using the exact general formula, we obtain exact formulas for the bias, mean squared error (MSE), skewness, and kurtosis of the FRR estimator. Since these formulas are very complex, we compare the bias, MSE, skewness, and kurtosis of the FRR estimator with those of ordinary least square (OLS) estimator by numerical evaluations. Our numerical results show that the range of MSE dominance of the FRR estimator over the OLS estimator is widen under a fat tail distributional assumption.     

Bias Correction in Generalized Nonlinear Models with Dispersion Covariates

We derive general formulae for the second-order biases of maximum likelihood estimates of the parameters in generalized nonlinear models with dispersion covariates. This result generalizes previous work by Botter and Cordeiro (1998 Botter , D. A. , Cordeiro , G. M. ( 1998 ). Improved estimates for generalized linear models with dispersion covariates . J. Statist. Comput. Simul. 62 : 91 – 104 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) and Cordeiro and McCullagh (1991 Cordeiro , G. M. , McCullagh , P. ( 1991 ). Bias correction in generalized linear models . J. Roy Statist. Soc. B 53 : 629 – 643 . [Google Scholar]). The practical use of such bias corrections is illustrated in a simulation study.     

ASYMPTOTIC EFFICIENCY OF ANALYSIS OF RAO'S QUADRATIC ENTROPY

Since Rao introduced the Quadratic Entropy (QE) in 1982, results on mathematical and statistical properties of the QE and its applications in data analysis and population indices have been published in the literature. In this paper, we study the asymptotic efficiency of the analysis of Rao's quadratic entropy (ANOQE) which is a generalization of the classical analysis of variance (ANOVA). Based on the results of Liu and Rao [1] Liu, Z. J. and Rao, C. R. 1995. Asymptotic distribution of statistics based on quadratic entropy and bootstrapping. JSPI, 43: 1–18.  [Google Scholar]and Liu [2] Liu, Z. J. 1991. Bootstrapping one way analysis of Rao's quadratic entropy. Comm. Statist., 20: 1683–1702.  [Google Scholar]on asymptotic distribution and the bootstrap of the ANOQE, we derive the Bahadur's asymptotic efficiency of the ANOQE and compare efficiency of ANOQE tests based on different QE's.