Confidence Sets Based on Thresholding Estimators in High-Dimensional Gaussian Regression Models

We study confidence intervals based on hard-thresholding, soft-thresholding, and adaptive soft-thresholding in a linear regression model where the number of regressors k may depend on and diverge with sample size n. In addition to the case of known error variance, we define and study versions of the estimators when the error variance is unknown. In the known-variance case, we provide an exact analysis of the coverage properties of such intervals in finite samples. We show that these intervals are always larger than the standard interval based on the least-squares estimator. Asymptotically, the intervals based on the thresholding estimators are larger even by an order of magnitude when the estimators are tuned to perform consistent variable selection. For the unknown-variance case, we provide nontrivial lower bounds and a small numerical study for the coverage probabilities in finite samples. We also conduct an asymptotic analysis where the results from the known-variance case can be shown to carry over asymptotically if the number of degrees of freedom n ? k tends to infinity fast enough in relation to the thresholding parameter.     

A Simple Test Procedure When Data are Given by n Indepenent Blocks

In this article we consider a test procedure which is useful in the situations where data are given by n independent blocks and the experimental conditions differ between blocks. The basic idea is very simple. The significance of the sample for each block is calculated and then standardized by its null mean and variance. The sum of standardized significances is proposed as a test statistic. The normal approximation for large n and the exact method for small n are applied in the continuous case. For the discrete case, some devices are also proposed. Several examples are given in order to explain how to apply the procedure.     

Constrained nonparametric maximum-likelihood estimation for mixture models

A nonparametric mixture model specifies that observations arise from a mixture distribution, ∫ f(x, θ) dG(θ), where the mixing distribution G is completely unspecified. A number of algorithms have been developed to obtain unconstrained maximum-likelihood estimates of G, but none of these algorithms lead to estimates when functional constraints are present. In many cases, there is a natural interest in functional ?(G), such as the mean and variance, of the mixing distribution, and profile likelihoods and confidence intervals for ?(G) are desired. In this paper we develop a penalized generalization of the ISDM algorithm of Kalbfleisch and Lesperance (1992) that can be used to solve the problem of constrained estimation. We also discuss its use in various different applications. Convergence results and numerical examples are given for the generalized ISDM algorithm, and asymptotic results are developed for the likelihood-ratio test statistics in the multinomial case.     

Preliminary Goodness-of-Fit Tests for Normality do not Validate the One-Sample Student t

One of the most basic topics in many introductory statistical methods texts is inference for a population mean, μ. The primary tool for confidence intervals and tests is the Student t sampling distribution. Although the derivation requires independent identically distributed normal random variables with constant variance, σ², most authors reassure the readers about some robustness to the normality and constant variance assumptions. Some point out that if one is concerned about assumptions, one may statistically test these prior to reliance on the Student t. Most software packages provide optional test results for both (a) the Gaussian assumption and (b) homogeneity of variance. Many textbooks advise only informal graphical assessments, such as certain scatterplots for independence, others for constant variance, and normal quantile–quantile plots for the adequacy of the Gaussian model. We concur with this recommendation. As convincing evidence against formal tests of (a), such as the Shapiro–Wilk, we offer a simulation study of the tails of the resulting conditional sampling distributions of the Studentized mean. We analyze the results of systematically screening all samples from normal, uniform, exponential, and Cauchy populations. This pretest does not correct the erroneous significance levels and makes matters worse for the exponential. In practice, we conclude that graphical diagnostics are better than a formal pretest. Furthermore, rank or permutation methods are recommended for exact validity in the symmetric case.     

Coverage accuracy for a mean without third moment

For constructing a confidence interval for the mean of a random variable with a known variance, one may prefer the sample mean standardized by the true standard deviation to the Student's t-statistic since the information of knowing the variance is used in the former way. In this paper, by comparing the leading error term in the expansion of the coverage probability, we show that the above statement is not true when the third moment is infinite. Our theory prefers the Student's t-statistic either when one-sided confidence intervals are considered for a heavier tail distribution or when two-sided confidence intervals are considered. Unlike other existing expansions for the Student's t-statistic, the derived explicit expansion for the case of infinite third moment can be used to estimate the coverage error so that bias correction becomes possible.     

Tests and confidence intervals for the shape parameter of a gamma distribution

Inference based on the Central Limit Theorem has only first order accuracy. We give tests and confidence intervals (CIs) of second orderaccuracy for the shape parameter ρ of a gamma distribution for both the unscaled and scaled cases.

Tests and CIs based on moment and cumulant estimates are considered as well as those based on the maximum likelihood estimate (MLE).

For the unscaled case the MLE is the moment estimate of order zero; the most efficient moment estimate of integral order is the sample mean, having asymptotic relative efficiency (ARE) .61 when ρ= 1.

For the scaled case the most efficient moment estimate is a functionof the mean and variance. Its ARE is .39 when ρ = 1.

Our motivation for constructing these tests of ρ = 1 and CIs forρ is to provide a simple and convenient method for testing whether a distribution is exponential in situations such as rainfall models where such an assumption is commonly made.     

A class of fixed-width confidence intervals for a ranked normal mean

Suppose that we are given k(≥ 2) independent and normally distributed populations π₁, …, π_k where π_i has unknown mean μ_i and unknown variance σ² _i (i = 1, …, k). Let μ_[i] (i = 1, …, k) denote the i^th smallest one of μ₁, …, μ_k. A two-stage procedure is used to construct lower and upper confidence intervals for μ_[i] and then use these to obtain a class of two-sided confidence intervals on μ_[i] with fixed width. For i = k, the interval given by Chen and Dudewicz (1976) is a special case. Comparison is made between the class of two-sided intervals and a symmetric interval proposed by Chen and Dudewicz (1976) for the largest mean, and it is found that for large values of k at least one of the former intervals requires a smaller total sample size. The tables needed to actually apply the procedure are provided.     

A modified one-sample test for goodness-of-fit

This paper introduces a modified one-sample test of goodness-of-fit based on the cumulative distribution function. Damico [A new one-sample test for goodness-of-fit. Commun Stat – Theory Methods. 2004;33:181–193] proposed a test for testing goodness-of-fit of univariate distribution that uses the concept of partitioning the probability range into n intervals of equal probability mass 1/n and verifies that the hypothesized distribution evaluated at the observed data would place one case into each interval. The present paper extends this notion by allowing for m intervals of probability mass r/n, where r≥1 and n=m×r. A simulation study for small and moderate sample sizes demonstrates that the proposed test for two observations per interval under various alternatives is more powerful than the test proposed by Damico (2004).     

Heterogeneity of variance and biased hypothesis tests

This study examined the influence of heterogeneity of variance on Type I error rates and power of the independent-samples Student's t-test of equality of means on samples of scores from normal and 10 non-normal distributions. The same test of equality of means was performed on corresponding rank-transformed scores. For many non-normal distributions, both versions produced anomalous power functions, resulting partly from the fact that the hypothesis test was biased, so that under some conditions, the probability of rejecting H ₀ decreased as the difference between means increased. In all cases where bias occurred, the t-test on ranks exhibited substantially greater bias than the t-test on scores. This anomalous result was independent of the more familiar changes in Type I error rates and power attributable to unequal sample sizes combined with unequal variances.     

Comparison of ANOVA-F and ANOM tests with regard to type I error rate and test power

A Monte Carlo simulation was conducted to compare the type I error rate and test power of the analysis of means (ANOM) test to the one-way analysis of variance F-test (ANOVA-F). Simulation results showed that as long as the homogeneity of the variance assumption was satisfied, regardless of the shape of the distribution, number of group and the combination of observations, both ANOVA-F and ANOM test have displayed similar type I error rates. However, both tests have been negatively affected from the heterogeneity of the variances. This case became more obvious when the variance ratios increased. The test power values of both tests changed with respect to the effect size (Δ), variance ratio and sample size combinations. As long as the variances are homogeneous, ANOVA-F and ANOM test have similar powers except unbalanced cases. Under unbalanced conditions, the ANOVA-F was observed to be powerful than the ANOM-test. On the other hand, an increase in total number of observations caused the power values of ANOVA-F and ANOM test approach to each other. The relations between effect size (Δ) and the variance ratios affected the test power, especially when the sample sizes are not equal. As ANOVA-F has become to be superior in some of the experimental conditions being considered, ANOM is superior in the others. However, generally, when the populations with large mean have larger variances as well, ANOM test has been seen to be superior. On the other hand, when the populations with large mean have small variances, generally, ANOVA-F has observed to be superior. The situation became clearer when the number of the groups is 4 or 5.     

Smooth tests of fit for censored gmma samples

In this paper properties of two estimators of Cpm are investigated in terms of changes in the process mean and variance. The bias and mean squared error of these estimators are derived. It can be shown that the estimate of Cpm proposed by Chan, Cheng and Spiring (1988) has smaller bias than the one proposed by Boyles (1991) and also has a smaller mean squared error under certain conditions. Various approximate confidence intervals for Cpm are obtained and are compared in terms of coverage probabilities, missed rate and average interval width.     

Confidence intervals for the mean of a normal distribution with restricted parameter space

For a normal distribution with known variance, the standard confidence interval of the location parameter is derived from the classical Neyman procedure. When the parameter space is known to be restricted, the standard confidence interval is arguably unsatisfactory. Recent articles have addressed this problem and proposed confidence intervals for the mean of a normal distribution where the parameter space is not less than zero. In this article, we propose a new confidence interval, rp interval, and derive the Bayesian credible interval and likelihood ratio interval for general restricted parameter space. We compare these intervals with the standard interval and the minimax interval. Simulation studies are undertaken to assess the performances of these confidence intervals.     

THE POWER OF STUDENT'S t TEST: CAN A NON-SIMILAR TEST DO BETTER?

Lehmann & Stein (1948) proved the existence of non-similar tests which can be more powerful than best similar tests. They used Student's problem of testing for a non-zero mean given a random sample from the normal distribution with unknown variance as an example. This raises the question: should we use a non-similar test instead of Student's t test? Questions like this can be answered by comparing the power of the test with the power envelope. This paper discusses the difficulties involved in computing power envelopes. It reports an empirical comparison of the power of the t test and the power envelope and finds that the two are almost identical especially for sample sizes greater than 20. These findings suggest that, as well as being uniformly most powerful (UMP) within the class of similar tests, Student's t test is approximately UMP within the class of all tests. For practical purposes it might also be regarded as UMP when moderate or large sample sizes are involved.     

A class of k-sample distribution-free tests for location against ordered alternatives

This paper introduces a new class of distribution-free tests for testing the homogeneity of several location parameters against ordered alternatives. The proposed class of test statistics is based on a linear combination of two-sample U-statistics based on subsample extremes. The mean and variance of the test statistic are obtained under the null hypothesis as well as under the sequence of local alternatives. The optimal weights are also determined. It is shown via Pitman ARE comparisons that the proposed class of test statistics performs better than its competitor tests in case of heavy-tailed and long-tailed distributions     

Analysis of Linear Models with One Factor Having Both Fixed and Random Levels

ABSTRACT

A general theory for a case where a factor has both fixed and random effect levels is developed under one-way treatment structure model. Estimation procedures for the fixed effects and variance components are consider for the model. The testing of fixed effects is considered when the variance–covariance matrix is known and unknown. Confidence intervals for estimable functions and prediction intervals for predictable functions are constructed. The computational procedures are illustrated using data from an on-farm trial.     

Variance Components Testing in ANOVA‐Type Mixed Models

The purpose of this article is threefold. First, variance components testing for ANOVA ‐type mixed models is considered, in which response may not be divided into independent sub‐vectors, whereas most of existing methods are for models where response can be divided into independent sub‐vectors. Second, testing that a certain subset of variance components is zero. Third, as normality is often violated in practice, it is desirable to construct tests under very mild assumptions. To achieve these goals, an adaptive difference‐based test and an adaptive trace‐based test are constructed. The test statistics are asymptotically normal under the null hypothesis, are consistent against all global alternatives and can detect local alternatives distinct from the null at a rate as close to n ^{? 1 ∕ 2} as possible with n being the sample size. Moreover, when the dimensions of variance components in different sets are bounded, we develop a test with chi‐square as its limiting null distribution. The finite sample performance of the tests is examined via simulations, and a real data set is analysed for illustration.     

Double threshold autoregressive conditionally heteroscedastic model building by genetic algorithms

Asymmetric behaviour in both mean and variance is often observed in real time series. The approach we adopt is based on double threshold autoregressive conditionally heteroscedastic (DTARCH) model with normal innovations. This model allows threshold nonlinearity in mean and volatility to be modelled as a result of the impact of lagged changes in assets and squared shocks, respectively. A methodology for building DTARCH models is proposed based on genetic algorithms (GAs). The most important structural parameters, that is regimes and thresholds, are searched for by GAs, while the remaining structural parameters, that is the delay parameters and models orders, vary in some pre-specified intervals and are determined using exhaustive search and an Asymptotic Information Criterion (AIC) like criterion. For each structural parameters trial set, a DTARCH model is fitted that maximizes the (penalized) likelihood (AIC criterion). For this purpose the iteratively weighted least squares algorithm is used. Then the best model according to the AIC criterion is chosen. Extension to the double threshold generalized ARCH (DTGARCH) model is also considered. The proposed methodology is checked using both simulated and market index data. Our findings show that our GAs-based procedure yields results that comparable to that reported in the literature and concerned with real time series. As far as artificial time series are considered, the proposed procedure seems to be able to fit the data quite well. In particular, a comparison is performed between the present procedure and the method proposed by Tsay [Tsay, R.S., 1989, Testing and modeling threshold autoregressive processes. Journal of the American Statistical Association, Theory and Methods, 84, 231–240.] for estimating the delay parameter. The former almost always yields better results than the latter. However, adopting Tsay's procedure as a preliminary stage for finding the appropriate delay parameter may save computational time specially if the delay parameter may vary in a large interval.     

Finite mixtures of natural exponential families

Let μ be a positive measure concentrated on R⁺ generating a natural exponential family (NEF) F with quadratic variance function V_F(m), m being the mean parameter of F. It is shown that v(dx) = (γ+x)μ(γ ≥ 0) (γ ≥ 0) generates a NEF G whose variance function is of the form l(m)Δ+cΔ(m), where l(m) is an affine function of m, Δ(m) is a polynomial in m (the mean of G) of degree 2, and c is a constant. The family G turns out to be a finite mixture of F and its length-biased family. We also examine the cases when F has cubic variance function and show that for suitable choices of γ the family G has variance function of the form P(m) + Q(m)m where P, Q are polynomials in m of degree m2 while Δ is an affine function of m. Finally we extend the idea to two dimensions by considering a bivariate Poisson and bivariate gamma mixture distribution.     

Extending Product Method of Estimation to Positive Correlation Case in Surveys

The product method of estimation (Murthy, 1964) complements the ratio method when the study variate, y, and an auxiliary variate, x, have negative correlation. However, such cases are not frequent in survey practice. This paper suggests a simple transformation of x in the more common situation of positive correlation between y and x, to permit a product method of estimation rather than a ratio method. This leads to the advantage that the bias and mean square error have exact expressions. The technique developed by Quenouille (1956) and applied by Shukla (1976) is used for making the estimator unbiased. The minimum variance situation is investigated. Two numerical examples are included. The case of negative correlation is also examined.     

Improved set estimators for the coefficients of a linear model when the error distribution is spherically symmetric with unknown variances

The usual confidence set for p (p ≥ 3) coefficients of a linear model is known to be dominated by the James-Stein confidence sets under the assumption of spherical symmetric errors with known variance (Hwang and Chen 1986). For the same confidence-set problem but for the unknown-variance case, naturally one replaces the unknown variance by an estimator. For the normal case, many previous studies have shown numerically that the resultant James-Stein confidence sets dominate the resultant usual confidence sets, i.e., the F confidence sets. In this paper we provide a further asymptotic justification, and we discover the same advantage of the James-Stein confidence sets for normal error as well as spherically symmetric error.