Adaptive Posterior Mode Estimation of a Sparse Sequence for Model Selection

Abstract.  For the problem of estimating a sparse sequence of coefficients of a parametric or non-parametric generalized linear model, posterior mode estimation with a Subbotin( λ , ν ) prior achieves thresholding and therefore model selection when ν   ∈    [0,1] for a class of likelihood functions. The proposed estimator also offers a continuum between the (forward/backward) best subset estimator ( ν  =  0 ), its approximate convexification called lasso ( ν  =  1 ) and ridge regression ( ν  =  2 ). Rather than fixing ν , selecting the two hyperparameters λ and ν adds flexibility for a better fit, provided both are well selected from the data. Considering first the canonical Gaussian model, we generalize the Stein unbiased risk estimate, SURE( λ , ν ), to the situation where the thresholding function is not almost differentiable (i.e. ν    1 ). We then propose a more general selection of λ and ν by deriving an information criterion that can be employed for instance for the lasso or wavelet smoothing. We investigate some asymptotic properties in parametric and non-parametric settings. Simulations and applications to real data show excellent performance.     

Lexigenesis in ancestral south-east Australian Aboriginal language

The 1/x frequency distribution is known to researchers ranging from economists and biologists to electronic engineers. It is known to linguists as Zipf's Law (Zipf, 1949) and has recently been shown not to be a consequence of the Central Limit Theorem (Troll & Graben, 1998)--leaving an "unsolved problem' in information theory (Jones, 1999). This 1/x distribution, associated with scale-invariant physical systems (Machlup & Hoshiko, 1980), is a special case of the general power law x^λ arising from the Lagrangian L(x,[Fdot](x)) = ½x^1-λ[Fdot]² and, as λ need not be an integer, some related research understandably involves fractals (Allison et al. , 2001). The present paper generalizes this Lagrangian to include a van der Waals effect. It is argued that ancestral Aboriginal language consisted of root-morphemes that were built up into, and often condensed within, subsequent words or lexemes. Using discrete-optimization techniques pioneered elsewhere (Illert, 1987; Reverberi, 1985), and the new morpho-statistics, this paper models lexeme-condensation in ancestral south-east Australian Aboriginal language.     

A Characterization of the Normal Family of Distributions Based on the Bias of the Maximum Likelihood Estimator

This paper characterizes the family of Normal distributions within the class of exponential families of distributions, via the structure of the bias of the maximum likelihood estimator Θ _n of the canonical parameter Θ . More specifically, when E _θ ( Θ n ) – Θ = (1/ n ) Q ( Θ ) + o (1/ n ), the equality Q ( Θ ) = 0 proves to be a property of the Normal distribution only. The same conclusion is obtained for the one-dimensional case bt assuming that Q ( Θ ) is a polynomial of Θ .     

Exact Slopes of Test Statistics for the Multivariate Exponential Family

The objective of this paper is to investigate exact slopes of test statistics { T_n } when the random vectors X ₁, ..., X_n are distributed according to an unknown member of an exponential family { P _θ; θ∈Ω. Here Ω is a parameter set. We will be concerned with the hypothesis testing problem of H ₀θ∈Ω₀ vs H ₁: θ∉Ω₀ where Ω₀ is a subset of Ω. It will be shown that for an important class of problems and test statistics the exact slope of { T_n } at η in Ω−Ω₀ is determined by the shortest Kullback–Leibler distance from {θ: T_n (λ(θ)) = T_n (λ(π))} to Ω₀, λ_θ = E _θ)( X ).     

A NOTE ON ESTIMATING THE LOCATION AND SCALE PARAMETERS OF THE EXPONENTIAL DISTRIBUTION FROM A CENSORED SAMPLE

   
Replacing f (x)/F (x) by α+β(x- θ)/σ in the maximum likelihood equations ∂L/∂θ and ∂L/∂σ calculated from a censored sample, a pair of estimators θ^e and σ^e, is obtained. The variances and covariances of these estimators are calculated and compared with the corresponding values for the best linear unbiassed (BLU) estimators.     

Residual analysis for spatial point processes (with discussion)

Summary.  We define residuals for point process models fitted to spatial point pattern data, and we propose diagnostic plots based on them. The residuals apply to any point process model that has a conditional intensity; the model may exhibit spatial heterogeneity, interpoint interaction and dependence on spatial covariates. Some existing ad hoc methods for model checking (quadrat counts, scan statistic, kernel smoothed intensity and Berman's diagnostic) are recovered as special cases. Diagnostic tools are developed systematically, by using an analogy between our spatial residuals and the usual residuals for (non-spatial) generalized linear models. The conditional intensity λ plays the role of the mean response. This makes it possible to adapt existing knowledge about model validation for generalized linear models to the spatial point process context, giving recommendations for diagnostic plots. A plot of smoothed residuals against spatial location, or against a spatial covariate, is effective in diagnosing spatial trend or co-variate effects. Q – Q -plots of the residuals are effective in diagnosing interpoint interaction.     

Semiparametric Likelihood Based Method for Goodness of Fit Tests and Estimation in Upgraded Mixture Models

We use Owen's (1988, 1990) empirical likelihood method in upgraded mixture models. Two groups of independent observations are available. One is z ₁, ..., z _n which is observed directly from a distribution F ( z ). The other one is x ₁, ..., x _m which is observed indirectly from F ( z ), where the x _i s have density ∫ p ( x | z ) dF ( z ) and p ( x | z ) is a conditional density function. We are interested in testing H ₀: p ( x | z ) = p ( x | z ; θ ), for some specified smooth density function. A semiparametric likelihood ratio based statistic is proposed and it is shown that it converges to a chi-squared distribution. This is a simple method for doing goodness of fit tests, especially when x is a discrete variable with finitely many values. In addition, we discuss estimation of θ and F ( z ) when H ₀ is true. The connection between upgraded mixture models and general estimating equations is pointed out.     

A Starship Estimation Method for the Generalized λ Distributions

A development of the 'starship' method (Owen, 1988), a computer intensive estimation method, is presented for two forms of generalized λ distributions (gλd). The method can be used for the full parameter space and is flexible, allowing choice of both the form of the generalized λ distribution and of the nature of fit required. Some examples of its use in fitting data and approximating distributions are given. Some simulation studies explore the sampling distribution of the parameter estimates produced by this method for selected values of the parameters and consider comparisons with two other methods, for one of the gλd distributional forms, not previously so investigated. In the forms and parameter regions available to the other methods, it is demonstrated that the starship compares favourably. Although the differences between the methods, where available, tend to disappear with largersamples, the parameter coverage, flexibility and adaptability of the starship method make it attractive. However, the paper also demonstrates that care is needed when fitting and using such quantile-defined distributional families that are rich in shape, but have complex properties.     

The Distribution of Sums of I.I.D. Pareto Random Variables with Arbitrary Shape Parameter

Laplace transforms are used to derive an exact expression for the cdf of the sum of n i.i.d. Pareto random variables with common pdf f(x) = (α/β)(1 + x/β)^?α?1 for x > 0, where α > 0 and is not an integer, and β > 0. An attractive feature of this expression is that it involves an integral of non oscillating real-valued functions on the positive real line. Examples of values of cdfs are provided and are compared to those determined via simulations.     

Characterization of laws by balancing weighting against a binary operation

LetL(X) be the law of a positive random variableX, andZ be positive and independent ofX. Solution pairs (L(X), L(Z)) are sought for the in-law equation $\hat X \cong X \circ Z$ where $L(\hat X)$ is a weighted law constructed fromL(X), and ° is a binary operation which in some sense is increasing. The class of weights includes length biasing of arbitrary order. When ° is the maximum operation a complete solution in terms of a product integral is found for arbitrary weighting. Examples are given. An identity for the length biasing operator is used when ° is multiplication to establish a general solution in terms of an already solved inverse equation. Some examples are given.     

COMBINING INFORMATION FROM LINEAR MODELS ON POSSIBLY DIFFERENT SCALES

This paper examines the joint statistical analysis of M independent data sets, the jth of which satisfies the model λ_j Y_j=X_jB +ε_j, where the λ_j are unknown and the ε_i are normally distributed with a known correlation structure. The maximum likelihood equations, their asymptotic covariance matrix, and the likelihood ratio test of the hypothesis that the λ_js are all equal are derived. These results are applied to two examples.     

On the effect of overdispersion on exact conditional tests

Let Y ₁, . . ., Y_n denote independent random variables such that Y_j has a one-parameter exponential family distribution with canonical parameter θ _j =λ+ψ X_j ; here X ₁, . . ., X_n are known constants. Consider a test of the null hypothesis ψ=0. Under the null hypothesis, A =Σ Y_j is sufficient for λ and, hence, a test of ψ=0 may be based on the conditional distribution of T =Σ X_j Y_j given A , which is independent of λ. In this paper, the effects of overdispersion due to a mixture model on the conditional distribution of T given A are considered.     

Non-parametric Regression with Dependent Censored Data

Abstract.  Let ( X _i , Y _i ) ( i = 1 ,…, n ) be n replications of a random vector ( X , Y  ), where Y is supposed to be subject to random right censoring. The data ( X _i , Y _i ) are assumed to come from a stationary α -mixing process. We consider the problem of estimating the function m ( x ) = E ( φ ( Y ) |  X = x ), for some known transformation φ . This problem is approached in the following way: first, we introduce a transformed variable     , that is not subject to censoring and satisfies the relation     , and then we estimate m ( x ) by applying local linear regression techniques. As a by-product, we obtain a general result on the uniform rate of convergence of kernel type estimators of functionals of an unknown distribution function, under strong mixing assumptions.     

MINIMAX ESTIMATION OF A MULTIVARIATE NORMAL MEAN UNDER A CONVEX LOSS FUNCTION

Let X = (X₁, - X_p)prime; ˜ N_p (μ, Σ) where μ= (μ₁, -, μ_p)' and Σ= diag (Σ²₁, -, Σ²_p) are both unknown and p3. Let (n_i - 2) w_i/Σ²_i! X²_ni, independent. of w_i (I ≠ j = 1, -, p). Assume that (w₁, -, w_p) and X are independent. Define W = diag (w₁, -, w_p) and ¶ X ¶²_w= X'W^-1Q^-1W^-1X where Q = diag (q₁, -,n q_p), q_i > 0, i = 1, -, p. In this paper, the minimax estimator of Berger & Bock (1976), given by δ (X, W) = [I_p - r(X, W) ¶ X ¶^-2_w Q^-1W^-1] X, is shown to be minimax relative to the convex loss (δ - μ)'[αQ + (1 - α) Σ^-1] δ - μ)/C, where C =α tr (Σ) + (1 - α)p and 0 α 1, under certain conditions on r(X, W). This generalizes the above mentioned result of Berger & Bock.     

Estimation of Integrated Volatility in Continuous-Time Financial Models with Applications to Goodness-of-Fit Testing

Abstract.  Properties of a specification test for the parametric form of the variance function in diffusion processes are discussed. The test is based on the estimation of certain integrals of the volatility function. If the volatility function does not depend on the variable x it is known that the corresponding statistics have an asymptotic normal distribution. However, most models of mathematical finance use a volatility function which depends on the state x . In this paper we prove that in the general case, where σ depends also on x the estimates of integrals of the volatility converge stably in law to random variables with a non-standard limit distribution. The limit distribution depends on the diffusion process X _t itself and we use this result to develop a bootstrap test for the parametric form of the volatility function, which is consistent in the general diffusion model.     

Nonparametric recursive estimation in stationary markov processes

This paper deals with a class of recursive kernel estimators of the transition probability density function t(y|x) of a stationary Markov process. A sufficient condition for such estimators to be weakly and strongly 2 consistent for almost all (x,y)∈R² is given. Further an L, convergence result is obtained. No continuity conditions are imposed on t(y|x).     

MINIMUM MSE ESTIMATION BY LINEAR COMBINATIONS OF ORDER STATISTICS1

In this paper we assume that in a random sample of size ndrawn from a population having the pdf f(x; θ) the smallest r₁ observations and the largest r₂ observations are censored (r₁0, r₂0). We consider the problem of estimating θ on the basis of the middle n-r₁-r₂ observations when either f(x;θ)=θ^-1f(x/θ) or f(x;θ) = (aθ)¹f(x-θ)/aθ) where f(·) is a known pdf, a (<0) is known and θ (>0) is unknown. The minimum mean square error (MSE) linear estimator of θ proposed in this paper is a “shrinkage” of the minimum variance linear unbiased estimator of θ. We obtain explicit expressions of these estimators and their mean square errors when (i) f(·) is the uniform pdf defined on an interval of length one and (ii) f(·) is the standard exponential pdf, i.e., f(x) = exp(–x), x0. Various special cases of censoring from the left (right) and no censoring are considered.     

Identifiability of distributions under competing risks and complementary risks model

Let X₁,X₂,…,X_p be p random variables with cdf's F₁(x),F₂(x),…,F_p(x)respectively. Let U = min(X₁,X₂,…,X_p) and V = max(X₁,X₂,…,X_p).In this paper we study the problem of uniquely determining and estimating the marginal distributions F₁,F₂,…,F_p given the distribution of U or of V.

First the problem of competing and complementary risks are introduced with examples and the corresponding identification problems are considered when the X₁'s are independently distributed and U(V) is identified, as well as the case when U(V) is not identified. The case when the X₁'s are dependent is considered next. Finally the problem of estimation is considered.     

Semiparametric Mixtures of Generalized Exponential Families

Abstract.  A semiparametric mixture model is characterized by a non-parametric mixing distribution Q (with respect to a parameter θ ) and a structural parameter β common to all components. Much of the literature on mixture models has focused on fixing β and estimating Q . However, this can lead to inconsistent estimation of both Q and the order of the model m . Creating a framework for consistent estimation remains an open problem and is the focus of this article. We formulate a class of generalized exponential family (GEF) models and establish sufficient conditions for the identifiability of finite mixtures formed from a GEF along with sufficient conditions for a nesting structure. Finite identifiability and nesting structure lead to the central result that semiparametric maximum likelihood estimation of Q and β fails. However, consistent estimation is possible if we restrict the class of mixing distributions and employ an information-theoretic approach. This article provides a foundation for inference in semiparametric mixture models, in which GEFs and their structural properties play an instrumental role.     

Efficient Bandwidth Selection in Non-parametric Regression

In this paper we use non-parametric local polynomial methods to estimate the regression function, m ( x ). Y may be a binary or continuous response variable, and X is continuous with non-uniform density. The main contributions of this paper are the weak convergence of a bandwidth process for kernels of order (0, k ), k =2 ^j , j ≥1 and the proposal of a local data-driven bandwidth selection method which is particularly beneficial for the case when X is not distributed uniformly. This selection method minimizes estimates of the asymptotic MSE and estimates the bias portion in an innovative way which relies on the order of the kernel and not estimation of m ²( x ) directly. We show that utilization of this method results in the achievement of the optimal asymptotic MSE by the estimator, i.e. the method is efficient. Simulation studies are provided which illustrate the method for both binary and continuous response cases.