Least squares and shrinkage estimation under bimonotonicity constraints

In this paper we describe active set type algorithms for minimization of a smooth function under general order constraints, an important case being functions on the set of bimonotone r×s matrices. These algorithms can be used, for instance, to estimate a bimonotone regression function via least squares or (a smooth approximation of) least absolute deviations. Another application is shrinkage estimation in image denoising or, more generally, regression problems with two ordinal factors after representing the data in a suitable basis which is indexed by pairs (i,j)∈{1,…,r}×{1,…,s}. Various numerical examples illustrate our methods.     

A COMPONENT OF VARIANCE DUE TO COMPETITION1

The existence of a component of variance for competition among sampling units or among individuals in a group was discussed by Yates and Zacopany in 1935. No procedure was suggested for estimating this component of variance. It is the purpose of this paper to give a procedure for estimating the component of variance due to competition and to apply the procedure to a set of data on weaning weights of pigs with 116 litters of various sizes and for Yorkshire, Chester-White, and Berkshire breeds. The first problem was to define litter size. Within this definition then, litters sizes of 3 to 14 pigs per litter were obtained. The variation among pigs within a litter of size h was considered to have an expected value equal to V_s+ V_ch where V_s is the sampling variance component and V_ch is the competition variance component for a litter of size h. In order to obtain an estimate of V_ch, a polynomial relation between h and V_ch was postulated. In particular, it was postulated that where E(.) denotes expected value. This form states that V_ch goes to zero for one pig per litter; it may be appropriate as long as small litter sizes (say 1 and 2 at least) are omitted from the analysis as was done in the present instance. Using an iterative procedure of reestimating the weights at each stage, a form of weighted least squares analysis was performed. The procedure appears to converge after three to four steps of iteration. Solutions for some or all of the parameters V_p, V_s, β₁ and β₂ for h even, h odd, and all h = 3, 4, …, 14, were obtained using among litter mean squares, A_h, only, using within litter mean squares, W_h, only, and using both A_h, and W_h values. The A_h values for h = 3, 4, and 5 appeared to form a different group than for the other values of h. Both the A_h and W_h mean squares were from 56 day weights adjusted for birth weight. The maximum value of V_ch for odd h, was nine whereas it was six for even h. Using all h the maximum value for V_ch occurred when h was equal to nine. It appeared that expressing V_ch as a quadratic function of litter size was satisfactory for these litter sizes and mean squares.     

Optimal designs for calibration of orientations

This paper concerns designed experiments involving observations of orientations following the models of Prentice (1989) and Rivest &Chang (2006). The authors state minimal conditions on the designs for consistent least squares estimation of the matrix parameters in these models. The conditions are expressed in terms of the axes and rotation angles of the design orientations. The authors show that designs satisfying U₁ + … + U_n = 0 are optimal in the sense of minimizing the estimation error average angular distance. The authors give constructions of optimal n‐point designs when n ≥ 4 and they compare the performance of several designs through approximations and simulation.     

Estimation of Conditional Ranks and Tests of Exogeneity in Nonparametric Nonseparable Models

Consider a nonparametric nonseparable regression model Y = ?(Z, U), where ?(Z, U) is strictly increasing in U and U ～ U[0, 1]. We suppose that there exists an instrument W that is independent of U. The observable random variables are Y, Z, and W, all one-dimensional. We construct test statistics for the hypothesis that Z is exogenous, that is, that U is independent of Z. The test statistics are based on the observation that Z is exogenous if and only if V = F_Y|Z(Y|Z) is independent of W, and hence they do not require the estimation of the function ?. The asymptotic properties of the proposed tests are proved, and a bootstrap approximation of the critical values of the tests is shown to be consistent and to work for finite samples via simulations. An empirical example using the U.K. Family Expenditure Survey is also given. As a byproduct of our results we obtain the asymptotic properties of a kernel estimator of the distribution of V, which equals U when Z is exogenous. We show that this estimator converges to the uniform distribution at faster rate than the parametric n^{? 1/2}-rate.     

Robust Linear Calibration

We regard the simple linear calibration problem where only the response y of the regression line y = β₀ + β₁ t is observed with errors. The experimental conditions t are observed without error. For the errors of the observations y we assume that there may be some gross errors providing outlying observations. This situation can be modeled by a conditionally contaminated regression model. In this model the classical calibration estimator based on the least squares estimator has an unbounded asymptotic bias. Therefore we introduce calibration estimators based on robust one-step-M-estimators which have a bounded asymptotic bias. For this class of estimators we discuss two problems: The optimal estimators and their corresponding optimal designs. We derive the locally optimal solutions and show that the maximin efficient designs for non-robust estimation and robust estimation coincide.     

Comparing the BLUEs Under Two Linear Models

In this article, we consider two linear models, ?₁ = {y, X β, V ₁} and ?₂ = {y, X β, V ₂}, which differ only in their covariance matrices. Our main focus lies on the difference of the best linear unbiased estimators, BLUEs, of X β under these models. The corresponding problems between the models {y, X β, I _n } and {y, X β, V}, i.e., between the OLSE (ordinary least squares estimator) and BLUE, are pretty well studied. Our purpose is to review the corresponding considerations between the BLUEs of X β under ?₁ and ?₂. This article is an expository one presenting also new results.     

Tests of regression coefficients under ridge regression models

This paper investigates two “non-exact” t-type tests, t( k₂) and t(k₂), of the individual coefficients of a linear regression model, based on two ordinary ridge estimators. The reported results are built on a simulation study covering 84 different models. For models with large standard errors, the ridge-based t-tests have correct levels with considerable gain in powers over those of the least squares t-test, t(0). For models with small standard errors, t(k₁) is found to be liberal and is not safe to use while, t(k₂) is found to slightly exceed the nominal level in few cases. When tie two ridge tests art: not winners, the results indicate that they don't loose much against t(0).     

Tree Structured QBD Markov Chains and Tree‐Like QBD Processes

Abstract

In this paper, we show that an arbitrary tree structured quasi‐birth–death (QBD) Markov chain can be embedded in a tree‐like QBD process with a special structure. Moreover, we present an algebraic proof that applying the natural fixed point iteration (FPI) to the nonlinear matrix equation V = B + ∑ _s=1 ^d U _s (I ? V)^?1 D _s that solves the tree‐like QBD process, is equivalent to the more complicated iterative algorithm presented by Yeung and Alfa (1996).     

Optimal Configuration of a Square Array Group Testing Algorithm

We consider the optimal configuration of a square array group testing algorithm (denoted A2) to minimize the expected number of tests per specimen. For prevalence greater than 0.2498, individual testing is shown to be more efficient than A2. For prevalence less than 0.2498, closed form lower and upper bounds on the optimal group sizes for A2 are given. Arrays of dimension 2 × 2, 3 × 3, and 4 × 4 are shown to never be optimal. The results are illustrated by considering the design of a specimen pooling algorithm for detection of recent HIV infections in Malawi.     

Maximum Likelihood Estimation in the Bivariate Binomial (0,1) Distribution:Application TO 2×2 Tables

We consider a 2×2 contingency table, with dichotomized qualitative characters (A,A) and (B,B), as a sample of size n drawn from a bivariate binomial (0,1) distribution. Maximum likelihood estimates p?₁p?₂ and p? are derived for the parameters of the two marginals p₁p₂ and the coefficient of correlation. It is found that p? is identical to Pearson's (1904)?=(χ²/n)½, where ?² is Pearson's usual chi-square for the 2×2 table. The asymptotic variance-covariance matrix of p?_lp?₂and p is also derived.     

CONDITIONING VS. STANDARDIZATION FOR CONTRASTS ON ERRORS ATTRACTED TO STABLE LAWS

In multiple regression and other settings one encounters the problem of estimating sampling distributions for contrast operations applied to i.i.d. errors. Permutation bootstrap applied to least squares residuals has been proven to consistently estimate conditionalsampling distributions of contrasts, conditional upon order statistics of errors, even for long-tailed error distributions. How does this compare with the unconditional sampling distribution of the contrast when standardizing by the sample s.d. of the errors (or the residuals)? For errors belonging to the domain of attraction of a normal we present a limit theorem proving that these distributions are far closer to one another than they are to the limiting standard normal distribution. For errors attracted to α-stable laws with α ≤ 2 we construct random variables possessing these conditional and unconditional sampling distributions and develop a Poisson representation for their a.s. limit correlation ρ_α. We prove that ρ₂= 1, ρ_α→ 1 for α → 0 + or 2 ?, and ρ_α< 1 a.s. for α < 2.     

On the Distribution of Matrix Quadratic Forms

A characterization of the distribution of the multivariate quadratic form given by X A X′, where X is a p × n normally distributed matrix and A is an n × n symmetric real matrix, is presented. We show that the distribution of the quadratic form is the same as the distribution of a weighted sum of non central Wishart distributed matrices. This is applied to derive the distribution of the sample covariance between the rows of X when the expectation is the same for every column and is estimated with the regular mean.     

Admissible Linear Estimators with Respect to Inequality Constraints

Admissibility of linear estimators is characterized in linear models E(Y)=Xβ, D(Y)=V, with an unknown multidimensional parameter (β, V) varying in the Cartesian product C × ν, where C is a subset of space and ν is a given set of non negative definite symmetric matrices. The relation between admissibility of inhomogeneous and homogeneous linear estimators is discussed, and some sufficient and necessary conditions for admissibility of an inhomogeneous linear estimator are given.     

A Test of Independence Based on the Likelihood of Cut-Points

We define a test statistic C _n based on the sum of the likelihood ratio statistics for testing independence in the 2 × 2 tables defined at n sample cut-points (X _i , Y _i ). The asymptotic distribution of C _n , given the cut-points, is sum of dependent χ² variables with one degree of freedom. We use the bootstrap to obtain the distribution of C _n . We compare the performance of several tests of bivariate independence, including Pearson, Spearman, and Kendall correlations, Blum-Kiefer-Rosenblatt statistic, and C _n under several copulas and given marginal distributions.     

Optimum invariant tests for random manova models

Consider the canonical-form MANOVA setup with X: n × p = (+ E, X_i n_i × p, i = 1, 2, 3, M_i: n_i × p, i = 1, 2, n₁ + n₂ + n₃) p, where E is a normally distributed error matrix with mean zero and dispersion I_n (> 0 (positive definite). Assume (in contrast with the usual case) that M₁i is normal with mean zero and dispersion I_n1) and M₂2 is either fixed or random normal with mean zero and different dispersion matrix I_n2 (being unknown. It is also assumed that M₁ E, and M₂ (if random) are all independent. For testing H₀) = 0 versus H₁: (> 0, it is shown that when either n₂ = 0 or M₂ is fixed if n₂ > 0, the trace test of Pillai (1955) is uniformly most powerful invariant (UMPI) if min(n₁, p)= 1 and locally best invariant (LBI) if min(n₁ p) > 1 underthe action of the full linear group Gl (p). When p > 1, the LBI test is also derived under a somewhat smaller group G_T(p) of p × p lower triangular matrices with positive diagonal elements. However, such results do not hold if n₂ > 0 and M₂ is random. The null, nonnull, and optimality robustness of Pillai's trace test under Gl(p) for suitable deviations from normality is pointed out.     

Influence measure for the L1 regression

Because outliers and leverage observations unduly affect the least squares regression, the identification of influential observations is considered an important and integrai part of the analysis. However, very few techniques have been developed for the residual analysis and diagnostics for the minimum sum of absolute errors, L₁ regression. Although the L₁ regression is more resistant to the outliers than the least squares regression, it appears that outliers (leverage) in the predictor variables may affect it. In this paper, our objective is to develop an influence measure for the L₁ regression based on the likelihood displacement function. We illustrate the proposed influence measure with examples.     

Testing homogeneity of several covariance matrices and multi-sample sphericity for high-dimensional data under non-normality

A test for homogeneity of g ? 2 covariance matrices is presented when the dimension, p, may exceed the sample size, n_i, i = 1, …, g, and the populations may not be normal. Under some mild assumptions on covariance matrices, the asymptotic distribution of the test is shown to be normal when n_i, p → ∞. Under the null hypothesis, the test is extended for common covariance matrix to be of a specified structure, including sphericity. Theory of U-statistics is employed in constructing the tests and deriving their limits. Simulations are used to show the accuracy of tests.     

Three estimators for the poisson regression model with measurement errors

We consider two consistent estimators for the parameters of the linear predictor in the Poisson regression model, where the covariate is measured with errors. The measurement errors are assumed to be normally distributed with known error variance σ _u ² . The SQS estimator, based on a conditional mean-variance model, takes the distribution of the latent covariate into account, and this is here assumed to be a normal distribution. The CS estimator, based on a corrected score function, does not use the distribution of the latent covariate. Nevertheless, for small σ _u ² , both estimators have identical asymptotic covariance matrices up to the order of σ _u ² . We also compare the consistent estimators to the naive estimator, which is based on replacing the latent covariate with its (erroneously) measured counterpart. The naive estimator is biased, but has a smaller covariance matrix than the consistent estimators (at least up to the order of σ _u ² ).     

SOME LAW OF THE ITERATED LOGARITHM TYPE RESULTS FOR THE EMPIRICAL PROCESS

We consider Z^±_n= sup_{0< t ≤ 1/2}2 U^±_n (t)/(t(1- t))^1/2, where + and -denote the positive and negative parts respectively of the sample paths of the empirical process U_n. U^±_n and U_n are seen to behave rather differently, which is tied to the asymmetry of the binomial distribution, or to the asymmetry of the distribution of small order statistics. Csáki (1975) showed that log Z^±_n/log₂n is the appropriate normalization for a law of the iterated logarithm (LIL) for Z^±_n we show that Z^-_n/(2 log₂n)^1/2 is the appropriate normalization for Z^-_n. Csörgö & Révész (1975) posed the question: if we replace the sup over (0,1/2) above, by -the sup over [a_n, 1-a_n] where a_n→0, how fast can a_n→0 and still have |Z_n|/(2 log₂n)^1/2 maintain a finite lim sup a.s.? This question is answered herein. The techniques developed are then used in Section 4 to give an interesting new proof of the upper class half of a result of Chung (1949) for |U_n(t)|. The proofs draw heavily on James (1975); two basic inequalities of that paper are strengthened to their potential, and are felt to be of independent interest.     

Joint Central Limit Theorem for Eigenvalue Statistics from Several Dependent Large Dimensional Sample Covariance Matrices with Application

Let X _n = (x _{i j} ) be a k ×n data matrix with complex‐valued, independent and standardized entries satisfying a Lindeberg‐type moment condition. We consider simultaneously R sample covariance matrices , where the Q _r 's are non‐random real matrices with common dimensions p ×k (k ≥p ). Assuming that both the dimension p and the sample size n grow to infinity, the limiting distributions of the eigenvalues of the matrices { B _{n r} } are identified, and as the main result of the paper, we establish a joint central limit theorem (CLT) for linear spectral statistics of the R matrices { B _{n r} }. Next, this new CLT is applied to the problem of testing a high‐dimensional white noise in time series modelling. In experiments, the derived test has a controlled size and is significantly faster than the classical permutation test, although it does have lower power. This application highlights the necessity of such joint CLT in the presence of several dependent sample covariance matrices. In contrast, all the existing works on CLT for linear spectral statistics of large sample covariance matrices deal with a single sample covariance matrix (R = 1).