Geodesic Monte Carlo on Embedded Manifolds

Markov chain Monte Carlo methods explicitly defined on the manifold of probability distributions have recently been established. These methods are constructed from diffusions across the manifold and the solution of the equations describing geodesic flows in the Hamilton–Jacobi representation. This paper takes the differential geometric basis of Markov chain Monte Carlo further by considering methods to simulate from probability distributions that themselves are defined on a manifold, with common examples being classes of distributions describing directional statistics. Proposal mechanisms are developed based on the geodesic flows over the manifolds of support for the distributions, and illustrative examples are provided for the hypersphere and Stiefel manifold of orthonormal matrices.     

A Restricted Subset Selection Rule for Selecting at Least One of the t Best Normal Populations in Terms of Their Means When Their Common Variance is Known,Case II

Consider k( ? 2) normal populations with unknown means μ₁, …, μ_k, and a common known variance σ². Let μ_[1] ? ??? ? μ_[k] denote the ordered μ_i.The populations associated with the t(1 ? t ? k ? 1) largest means are called the t best populations. Hsu and Panchapakesan (2004) proposed and investigated a procedure R_HPfor selecting a non empty subset of the k populations whose size is at most m(1 ? m ? k ? t) so that at least one of the t best populations is included in the selected subset with a minimum guaranteed probability P* whenever μ_{[k ? t + 1]} ? μ_{[k ? t]} ? δ*, where P*?and?δ* are specified in advance of the experiment. This probability requirement is known as the indifference-zone probability requirement. In the present article, we investigate the same procedure R_HP for the same goal as before but when k ? t < m ? k ? 1 so that at least one of the t best populations is included in the selected subset with a minimum guaranteed probability P* whatever be the configuration of the unknown μ_i. The probability requirement in this latter case is termed the subset selection probability requirement. Santner (1976) proposed and investigated a different procedure (R_S) based on samples of size n from each of the populations, considering both cases, 1 ? m ? k ? t and k ? t < m ? k. The special case of t = 1 was earlier studied by Gupta and Santner (1973) and Hsu and Panchapakesan (2002) for their respective procedures.     

Selecting the normal population with the smallest variance: A restricted subset selection rule

Consider k( ? 2) normal populations whose means are all known or unknown and whose variances are unknown. Let σ²_[1] ? ??? ? σ_[k]² denote the ordered variances. Our goal is to select a non empty subset of the k populations whose size is at most m(1 ? m ? k ? 1) so that the population associated with the smallest variance (called the best population) is included in the selected subset with a guaranteed minimum probability P* whenever σ²_[2]/σ_[1]² ? δ* > 1, where P* and δ* are specified in advance of the experiment. Based on samples of size n from each of the populations, we propose and investigate a procedure called R_BCP. We also derive some asymptotic results for our procedure. Some comparisons with an earlier available procedure are presented in terms of the average subset sizes for selected slippage configurations based on simulations. The results are illustrated by an example.     

On the spectrum of [v,k, t] trades

A [v, k, t] trade of volume m consists of two disjoint collections T₁ and T₂, each of m k-subsets of a v-set V, such that each t-subset of V is contained in the same number of blocks of T₁ and T₂, and each element of V is contained in at least one block of T₁. We study [v, k, t] trades, and investigate their spectrum (i.e., the collections of allowable volumes), using both theoretical techniques and computer-based searches.     

Optimal linear projections for enhancing desired data statistics

Problems involving high-dimensional data, such as pattern recognition, image analysis, and gene clustering, often require a preliminary step of dimension reduction before or during statistical analysis. If one restricts to a linear technique for dimension reduction, the remaining issue is the choice of the projection. This choice can be dictated by desire to maximize certain statistical criteria, including variance, kurtosis, sparseness, and entropy, of the projected data. Motivations for such criteria comes from past empirical studies of statistics of natural and urban images. We present a geometric framework for finding projections that are optimal for obtaining certain desired statistical properties. Our approach is to define an objective function on spaces of orthogonal linear projections—Stiefel and Grassmann manifolds, and to use gradient techniques to optimize that function. This construction uses the geometries of these manifolds to perform the optimization. Experimental results are presented to demonstrate these ideas for natural and facial images.     

Quadratic subspaces and construction of Bayes invariant quadratic estimators of variance components in mixed linear models

Gnot et al. (J Statist Plann Inference 30(1):223–236, 1992) have presented the formulae for computing Bayes invariant quadratic estimators of variance components in normal mixed linear models of the form where the matrices V _i , 1 ≤ i ≤ k − 1, are symmetric and nonnegative definite and V _k is an identity matrix. These formulae involve a basis of a quadratic subspace containing MV ₁ M,...,MV _k-1 M,M, where M is an orthogonal projector on the null space of X′. In the paper we discuss methods of construction of such a basis. We survey Malley’s algorithms for finding the smallest quadratic subspace including a given set of symmetric matrices of the same order and propose some modifications of these algorithms. We also consider a class of matrices sharing some of the symmetries common to MV ₁ M,...,MV _k-1 M,M. We show that the matrices from this class constitute a quadratic subspace and describe its explicit basis, which can be directly used for computing Bayes invariant quadratic estimators of variance components. This basis can be also used for improving the efficiency of Malley’s algorithms when applied to finding a basis of the smallest quadratic subspace containing the matrices MV ₁ M,...,MV _k-1 M,M. Finally, we present the results of a numerical experiment which confirm the potential usefulness of the proposed methods. Dedicated to the memory of Professor Stanisław Gnot.     

A series of group divisible designs

Bose and Shrikhande C19763 proved that if D(m, k, ?) is a Baer subdesign of another SBIBD D₁ (v₁, k₁ ?), k₁>k, then it also contains a complementary subdesign D^* which is symmetric GDD, D^* (v^*, k^*; ?-1, ?; m, n). Utilising this, we give a necessary condition for a SBIBD D to be a Baer subdesign of D₁ and also give the parameters. Some GD designs are constructed.     

Reliability and Some Characteristics of Circular Consecutive-k-Within-m-Out-of-n:F System

A system can be classified with respect to the physical arrangement of its components and the functioning principle. A circular consecutive k-within-m-out-of-n:F system consists of n circularly ordered components and fails if and only if there are m consecutive components that include among them at least k failed components. A circular consecutive k-within-m-out-of-n:F system turns into circular consecutive k-out-of-n:F for m = k and k-out-of-n:F system for m = n. In this study, signature-based analysis of circular consecutive k-within-m-out-of-n:F system is performed. A new approximation to this system is provided based on maximum number of failed components and an illustrative example is given for different values of n, m, k to compare the approximate results with simulated and exact results.     

Optimal linear combinations using householder transformations

In pattern classification of sampled vector valued random variables it is often essential, due to computational and accuracy considerations, to consider certain measurable transformations of the random variable. These transformations are generally of a dimension-reducing nature. In this paper we consider the class of linear dimension reducing transformations, i.e., the k × n matrices of rank k where k < n and n is the dimension of the range of the sampled vector random variable.

In this connection, we use certain results (Decell and Quirein, 1973), that guarantee, relative to various class separability criteria, the existence of an extremal transformation. These results also guarantee that the extremal transformation can be expressed in the form (I_k∣ Z)U where I_k is the k × k identity matrix and U is an orthogonal n × n matrix. These results actually limit the search for the extremal linear transformation to a search over the obviously smaller class of k × n matrices of the form (I_k ∣Z)U. In this paper these results are refined in the sense that any extremal transformation can be expressed in the form (I_K∣Z)H_p … H₁ where p ≤ min{k, n?k} and H_i is a Householder transformation i=l,…, p, The latter result allows one to construct a sequence of transformations (L_K∣ Z)H₁, (I_K Z)H₂H₁ … such that the values of the class separability criterion evaluated at this sequence is a bounded, monotone sequence of real numbers. The construction of the i-th element of the sequence of transformations requires the solution of an n-dimensional optimization problem. The solution, for various class separability criteria, of the optimization problem will be the subject of later papers. We have conjectured (with supporting theorems and empirical results) that, since the bounded monotone sequence of real class separability values converges to its least upper bound, this least upper bound is an extremal value of the class separability criterion.

Several open questions are stated and the practical implications of the results are discussed.     

On a conjecture of Kakosyan,Klebanov and Melamed

Suppose X₁, X₂, ..., X_m is a random sample of size m from a population with probability density function f(x), x>0 and let X_1,m<...m,m be the corresponding order statistics. We assume m as an integer valued random variable with P(m=k)=p(1?p)^k?1, k=1, 2, ... and 0 and n X_1,n for fixed n characterizes the exponential distribution. In this paper we prove that under the assumption of monotone hazard rate the identical distribution of and (n?r+1) (X_r,n?X_r?1,n) for some fixed r and n with 1≤r≤n, n≥2, X_0,n=0, characterizes the exponential distribution. Under the assumption of monotone hazard rate the conjecture of Kakosyan, Klebanov and Melamed follows from the above result with r=1.     

Asymptotics for multisample statistics

Consider a family of square-integrable R^d-valued statistics S_k = S_k(X_1,k1; X_2,k2;…; X_m,km), where the independent samples X_i,kj respectively have k_i i.i.d. components valued in some separable metric space X_i. We prove a strong law of large numbers, a central limit theorem and a law of the iterated logarithm for the sequence {S_k}, including both the situations where the sample sizes tend to infinity while m is fixed and those where the sample sizes remain small while m tends to infinity. We also obtain two almost sure convergence results in both these contexts, under the additional assumption that S_k is symmetric in the coordinates of each sample X_i,kj. Some extensions to row-exchangeable and conditionally independent observations are provided. Applications to an estimator of the dimension of a data set and to the Henze-Schilling test statistic for equality of two densities are also presented.     

Normal approximation in an urn model with indistinguishable balls

For the Bose-Einstein Statistics, where n indistinguishable balls are distributed in m urns such that all the arrangements are equally likely, define the random variables

M_k = number of urns containing exactly k balls each;

N_k = number of urns containing at least k balls each.

We consider the approximation of the distributions of M_k and N_k by suitable normal distributions, for large but finite m. Estimates are found for the error in the approximation to both the probability mass function and the distribution function in each case. These results apply also to the alternative model where no urn is allowed to be empty. The results are illustrated by some numerical examples.     

On a differential equation approach to the weighted orthogonal Procrustes problem

The weighted orthogonal Procrustes problem, an important class of data matching problems in multivariate data analysis, is reconsidered in this paper. It is shown that a steepest descent flow on the manifold of orthogonal matrices can naturally be formulated. This formulation has two important implications: that the weighted orthogonal Procrustes problem can be solved as an initial value problem by any available numerical integrator and that the first order and the second order optimality conditions can also be derived. The proposed approach is illustrated by numerical examples.     

On selecting a subset which contains all populations better than a control

Let π₁…, π_k denote k(≥ 2) populations with unknown means μ₁ , …, μ_k and variances σ₁ ² , …, σ_k ² , respectively and let π_o denote the control population having mean μ_o and variance σ_o ² . It is assumed that these populations are normally distributed with correlation matrix {ρ_ij}. The goal is to select a subset, of populations of π₁ , …, π_k which contains all the populations with means larger than or equal to the mean of the control one. Procedures are given for selecting such a subset so that the probability that all the populations with means larger than or equal to the mean of the control one are included in the selected subset is at least equal to a predetermined value P?(l/k < P? < 1). The goal treated here is a first step screening procedure that allows the experimenter to choose a subset and withhold judgement about which one has the largest mean. Then, if the one with the largest mean is desired it can be chosen from the selected subset on the basis of cost and other considerations. Percentage points are also included.     

Exact distributions associated with an i×j×k contingency table

In this paper an exact distributional framework is developed for analysing an IxJxK contingency table. It is shown that for the case of hypotheses H₀:P_ijk=P_i..P._j./K and H₀:P_ijk =P_i..P._j.P.._k the exact distributional results do not follow as simple extensions of the corresponding results obtained for an I×J table under the hypothesis of independence. From the factorial moment generating functions, expressions for the covariance matrices in terms of the Kronecker products of matrices, are presented. These expressions give indications whether or not Pearson's chi-square statistic should be corrected by the factor (n?1)/n or not. Marginal and conditional distributions are considered briefly and important differences with regard to the resuits for marginal and conditional distributions for an IxJ table are mentioned.     

DINDSCAL: direct INDSCAL

The well-known INDSCAL model for simultaneous metric multidimensional scaling (MDS) of three-way data analyzes doubly centered matrices of squared dissimilarities. An alternative approach, called for short DINDSCAL, is proposed for analyzing directly the input matrices of squared dissimilarities. An important consequence is that missing values can be easily handled. The DINDSCAL problem is solved by means of the projected gradient approach. First, the problem is transformed into a gradient dynamical system on a product matrix manifold (of Stiefel sub-manifold of zero-sum matrices and non-negative diagonal matrices). The constructed dynamical system can be numerically integrated which gives a globally convergent algorithm for solving the DINDSCAL. The DINDSCAL problem and its solution are illustrated by well-known data routinely used in metric MDS and INDSCAL. Alternatively, the problem can also be solved by iterative algorithm based on the conjugate (projected) gradient method, which MATLAB implementation is enclosed as an appendix.     

A two stage procedure for selecting δ∗-optimal means in the normal model

Suppose π₁,…,π_k are k normal populations with π_i having unknown mean μ_i and unknown variance σ². The population π_i will be called δ^?-optimal (or good) if μ_i is within a specified amountδ? of the largest mean. A two stage procedure is proposed which selects a subset of the k populations and guarantees with probability at least P^? that the selected subset contains only δ?-optimal π_i ’s. In addition to screening out non-good populations the rule guarantees a high proportion of sufficiently good π_i’S will be selected.     

Local asymptotic normality of intermediate and central order statistics

Consider n independent random variables Z_i,…, Z_n on R with common distribution function F, whose upper tail belongs to a parametric family F(t) = F_θ(t),t ≥ x₀, where θ ∈ ? ? R ^d. A necessary and sufficient condition for the family F_θ, θ ∈ ?, is established such that the k-th largest order statistic Z_n?k+1:n alone constitutes the central sequence yielding local asymptotic normality ( LAN ) of the loglikelihood ratio of the vector (Z_n?i+1:n)¹ _i=kof the k largest order statistics. This is achieved for k = k(n)→_n→∞∞ with k/n→_n→∞ 0.

In the case of vectors of central order statistics ( Z_r:n, Z_r+1:n,…, Z_s:n ), with r/n and s/n both converging to q ∈ ( 0,1 ), it turns out that under fairly general conditions any order statistic Z_m:n with r ≤ m ≤s builds the central sequence in a pertaining LAN expansion.These results lead to asymptotically optimal tests and estimators of the underlying parameter, which depend on single order statistics only     

The ℓ1 oblique procrustes problem

In this paper, we reconsider the well-known oblique Procrustes problem where the usual least-squares objective function is replaced by a more robust discrepancy measure, based on the ₁ norm or smooth approximations of it.We propose two approaches to the solution of this problem. One approach is based on convex analysis and uses the structure of the problem to permit a solution to the ₁ norm problem. An alternative approach is to smooth the problem by working with smooth approximations to the ₁ norm, and this leads to a solution process based on the solution of ordinary differential equations on manifolds. The general weighted Procrustes problem (both orthogonal and oblique) can also be solved by the latter approach. Numerical examples to illustrate the algorithms which have been developed are reported and analyzed.