Generalized minimum distance estimators of a linear model with correlated errors

R^p of a linear regression model of the type Y = Xθ + ɛ, where X is the design matrix, Y the vector of the response variable and ɛ the random error vector that follows an AR(1) correlation structure. These estimators are asymptotically analyzed, by proving their strong consistency, asymptotic normality and asymptotic efficiency. In a simulation study, a better behaviour of the Mean Squared Error of the proposed estimator with respect to that of the generalized least squares estimators is observed. Received: November 16, 1998; revised version: May 10, 2000     

Prediction based on linear combinations of order statistics and bivariate concomitants in the case of multivariate elliptical distributions

In this paper, by considering a (3n+1) -dimensional random vector (X₀, X^T, Y^T, Z^T)^T having a multivariate elliptical distribution, we derive the exact joint distribution of (X₀, a^TX_(n), b^TY_[n], c^TZ_[n])^T, where a, b, c∈?ⁿ, X_(n)=(X₍₁₎, …, X_(n))^T, X₍₁₎<···<X_(n), is the vector of order statistics arising from X, and Y_[n]=(Y_[1], …, Y_[n])^T and Z_[n]=(Z_[1], …, Z_[n])^T denote the vectors of concomitants corresponding to X_(n) ((Y_[r], Z_[r])^T, for r=1, …, n, is the vector of bivariate concomitants corresponding to X_(r)). We then present an alternate approach for the derivation of the exact joint distribution of (X₀, X_(r), Y_[r], Z_[r])^T, for r=1, …, n. We show that these joint distributions can be expressed as mixtures of four-variate unified skew-elliptical distributions and these mixture forms facilitate the prediction of X_(r), say, based on the concomitants Y_[r] and Z_[r]. Finally, we illustrate the usefulness of our results by a real data.     

Regression analysis using order statistics

In this article, we study the joint distribution of X and two linear combinations of order statistics, a ^T Y ₍₂₎ and b ^T Y ₍₂₎, where a = (a ₁, a ₂) ^T and b = (b ₁, b ₂) ^T are arbitrary vectors in R ² and Y ₍₂₎ = (Y ₍₁₎, Y ₍₂₎) ^T is a vector of ordered statistics obtained from (Y ₁, Y ₂) ^T when (X, Y ₁, Y ₂) ^T follows a trivariate normal distribution with a positive definite covariance matrix. We show that this distribution belongs to the skew-normal family and hence our work is a generalization of Olkin and Viana (J Am Stat Assoc 90:1373–1379, 1995) and Loperfido (Test 17:370–380, 2008).     

A stochastic Newton-Raphson method

A stochastic approximation procedure of the Robbins-Monro type is considered. The original idea behind the Newton-Raphson method is used as follows. Given n approximations X₁,…, X_n with observations Y₁,…, Y_n, a least squares line is fitted to the points (X_m, Y_m),…, (X_n, Y_n) where m<n may depend on n. The (n+1)st approximation is taken to be the intersection of the least squares line with y=0. A variation of the resulting process is studied. It is shown that this process yields a strongly consistent sequence of estimates which is asymptotically normal with minimal asymptotic variance.     

On the asymptotic expectation and variance of the number of boundary crossings

Let X₁,X₂,… be independent and identically distributed nonnegative random variables with mean μ, and let S_n = X₁ + … + X_n. For each λ > 0 and each n ≥ 1, let A_n be the interval [λn^Y, ∞), where γ > 1 is a constant. The number of times that S_n is in A_n is denoted by N. As λ tends to zero, the asymtotic behavior of N is studied. Specifically under suitable conditions, the expectation of N is shown to be (μλ^?1)^β + o(λ^?β/2 where β = 1/(γ-1) and the variance of N is shown to be (μλ^?1)^β(βμ¹)²σ² + o(λ^?β) where σ² is the variance of X_n.     

Two graphical displays for the detection of potentially influential subsets in regression

In the context of the general linear model Y=Xβ+ε, the matrix P_z =Z(Z^TZ)^?1 Z^T , where Z=(X: Y), plays an important role in determining least squares results. In this article we propose two graphical displays for the off-diagonal as well as the diagonal elements of P_Z . The two graphs are based on simple ideas and are useful in the detection of potentially influential subsets of observations in regression. Since P_Z is invariant with respect to permutations of the columns of Z, an added advantage of these graphs is that they can be used to detect outliers in multivariate data where the rows of Z are usually regarded as a random sample from a multivariate population. We also suggest two calibration points, one for the diagonal elements of P_Z and the other for the off-diagonal elements. The advantage of these calibration points is that they take into consideration the variability of the off-diagonal as well as the diagonal elements of P_Z . They also do not suffer from masking.     

Nonlinear regression in a trivariate elliptical distribution via a linear combination of order statistics

In this paper, by assuming that (X, Y ₁, Y ₂)^T has a trivariate elliptical distribution, we derive the exact joint distribution of X and a linear combination of order statistics from (Y ₁, Y ₂)^T and show that it is a mixture of unified bivariate skew-elliptical distributions. We then derive the corresponding marginal and conditional distributions for the special case of t kernel. We also present these results for an exchangeable case with t kernel and illustrate the established results with an air-pollution data.     

The T-type estimate of a class of partially non linear models

Abstract

This paper considers a partially non linear model E(Y|X, z, t) = f(X, β) + z^Tg(t) and gives its T-type estimate, which is a weighted quasi-likelihood estimate using sieve method and can be obtained by EM algorithm. The influence functions and asymptotic properties of T-type estimate (consistency and asymptotic normality) are discussed, and convergence rate of both parametric and non parametric components are obtained. Simulation results show the shape of influence functions and prove that the T-type estimate performs quite well. The proposed estimate is also applied to a data set and compared with the least square estimate and least absolute deviation estimate.     

Bounds for Expectations of Differences of Generalized Order Statistics Based on General and Life Distributions

Let X _? ^(r), r ≥ 1, denote generalized order statistics based on an arbitrary distribution function F with finite pth absolute moment for some 1 ≤ p ≤ ∞. We present sharp upper bounds on E(X _? ^(s) ? X _? ^(r)), 1 ≤ r < s, for F being either general or life distribution. The bounds are expressed in various scale units generated by pth central absolute or raw moments of F, respectively. The distributions achieving the bounds are specified.     

New sensitivity analysis subordinated to a contrast

ABSTRACT

In a model of the form Y = h(X₁, …, X_d) where the goal is to estimate a parameter of the probability distribution of Y, we define new sensitivity indices which quantify the importance of each variable X_i with respect to this parameter of interest. The aim of this paper is to define goal oriented sensitivity indices and we will show that Sobol indices are sensitivity indices associated to a particular characteristic of the distribution Y. We name the framework we present as Goal Oriented Sensitivity Analysis (GOSA).     

Model fitting and inference under latent equilibrium processes

This paper presents a methodology for model fitting and inference in the context of Bayesian models of the type f(Y | X,θ)f(X|θ)f(θ), where Y is the (set of) observed data, θ is a set of model parameters and X is an unobserved (latent) stationary stochastic process induced by the first order transition model f(X ^(t+1)|X ^(t),θ), where X ^(t) denotes the state of the process at time (or generation) t. The crucial feature of the above type of model is that, given θ, the transition model f(X ^(t+1)|X ^(t),θ) is known but the distribution of the stochastic process in equilibrium, that is f(X|θ), is, except in very special cases, intractable, hence unknown. A further point to note is that the data Y has been assumed to be observed when the underlying process is in equilibrium. In other words, the data is not collected dynamically over time. We refer to such specification as a latent equilibrium process (LEP) model. It is motivated by problems in population genetics (though other applications are discussed), where it is of interest to learn about parameters such as mutation and migration rates and population sizes, given a sample of allele frequencies at one or more loci. In such problems it is natural to assume that the distribution of the observed allele frequencies depends on the true (unobserved) population allele frequencies, whereas the distribution of the true allele frequencies is only indirectly specified through a transition model. As a hierarchical specification, it is natural to fit the LEP within a Bayesian framework. Fitting such models is usually done via Markov chain Monte Carlo (MCMC). However, we demonstrate that, in the case of LEP models, implementation of MCMC is far from straightforward. The main contribution of this paper is to provide a methodology to implement MCMC for LEP models. We demonstrate our approach in population genetics problems with both simulated and real data sets. The resultant model fitting is computationally intensive and thus, we also discuss parallel implementation of the procedure in special cases.     

Quantile inference for heteroscedastic regression models

Consider the nonparametric heteroscedastic regression model Y=m(X)+σ(X)?, where m(·) is an unknown conditional mean function and σ(·) is an unknown conditional scale function. In this paper, the limit distribution of the quantile estimate for the scale function σ(X) is derived. Since the limit distribution depends on the unknown density of the errors, an empirical likelihood ratio statistic based on quantile estimator is proposed. This statistics is used to construct confidence intervals for the variance function. Under certain regularity conditions, it is shown that the quantile estimate of the scale function converges to a Brownian motion and the empirical likelihood ratio statistic converges to a chi-squared random variable. Simulation results demonstrate the superiority of the proposed method over the least squares procedure when the underlying errors have heavy tails.     

Dispersive ordering—Some applications and examples

A basic concept for comparing spread among probability distributions is that of dispersive ordering. Let X and Y be two random variables with distribution functions F and G, respectively. Let F ⁻¹ and G ⁻¹ be their right continuous inverses (quantile functions). We say that Y is less dispersed than X (Y≤ _disp X) if G ⁻¹(β)−G ⁻¹(α)≤F ⁻¹(β)−F ⁻¹(α), for all 0<α≤β<1. This means that the difference between any two quantiles of G is smaller than the difference between the corresponding quantiles of F. A consequence of Y≤ _disp X is that |Y ₁−Y ₂| is stochastically smaller than |X ₁−X ₂| and this in turn implies var(Y)≤var(X) as well as E[|Y ₁−Y ₂|]≤E[|X ₁−X ₂|], where X ₁, X ₂ (Y ₁, Y ₂) are two independent copies of X(Y). In this review paper, we give several examples and applications of dispersive ordering in statistics. Examples include those related to order statistics, spacings, convolution of non-identically distributed random variables and epoch times of non-homogeneous Poisson processes. This work was supported in part by KOSEF through Statistical Research Center for Complex Systems at Seoul National University. Subhash Kochar is thankful to Dr. B. Khaledi for many helpful discussions.     

On Thresholds of Moving Averages with Given On-Target Significant Levels

Let X ₁, X ₂,… be a sequence of independent and identically distributed random variables, and let Y _n , n = K, K + 1, K + 2,… be the corresponding backward moving average of order K. At epoch n ≥ K, the process Y _n will be off target by the input X _n if it exceeds a threshold. By introducing a two-state Markov chain, we define a level of significance (1 ? a)% to be the percentage of times that the moving average process stays on target. We establish a technique to evaluate, or estimate, a threshold, to guarantee that {Y _n } will stay (1 ? a)% of times on target, for a given (1 ? a)%. It is proved that if the distribution of the inputs is exponential or normal, then the threshold will be a linear function in the mean of the distribution of inputs μ _X . The slope and intercept of the line, in each case, are specified. It is also observed that for the gamma inputs, the threshold is merely linear in the reciprocal of the scale parameter. These linear relationships can be easily applied to estimate the desired thresholds by samples from the inputs.     

Confidence Intervals for Nonparametric Regression Functions with Missing Data

Suppose that we have a nonparametric regression model Y = m(X) + ε with X ∈ R^p, where X is a random design variable and is observed completely, and Y is the response variable and some Y-values are missing at random. Based on the “complete” data sets for Y after nonaprametric regression imputation and inverse probability weighted imputation, two estimators of the regression function m(x₀) for fixed x₀ ∈ R^p are proposed. Asymptotic normality of two estimators is established, which is used to construct normal approximation-based confidence intervals for m(x₀). We also construct an empirical likelihood (EL) statistic for m(x₀) with limiting distribution of χ²₁, which is used to construct an EL confidence interval for m(x₀).     

ROBUST RIDGE REGRESSION BASED ON AN M-ESTIMATOR

Consider the linear regression model y =β₀1 +Xβ+ in the usual notation. It is argued that the class of ordinary ridge estimators obtained by shrinking the least squares estimator by the matrix (X¹X + kI)^-1X'X is sensitive to outliers in the ^variable. To overcome this problem, we propose a new class of ridge-type M-estimators, obtained by shrinking an M-estimator (instead of the least squares estimator) by the same matrix. Since the optimal value of the ridge parameter k is unknown, we suggest a procedure for choosing it adaptively. In a reasonably large scale simulation study with a particular M-estimator, we found that if the conditions are such that the M-estimator is more efficient than the least squares estimator then the corresponding ridge-type M-estimator proposed here is better, in terms of a Mean Squared Error criteria, than the ordinary ridge estimator with k chosen suitably. An example illustrates that the estimators proposed here are less sensitive to outliers in the y-variable than ordinary ridge estimators.     

Information Properties for Concomitants of Order Statistics in Farlie–Gumbel–Morgenstern (FGM) Family

Let (X _i , Y _i ), i = 1, 2,…, n be independent and identically distributed random variables from some continuous bivariate distribution. If X _(r) denotes the rth-order statistic, then the Y's associated with X _(r) denoted by Y _[r] is called the concomitant of the rth-order statistic. In this article, we derive an analytical expression of Shannon entropy for concomitants of order statistics in FGM family. Applying this expression for some well-known distributions of this family, we obtain the exact form of Shannon entropy, some of the information properties, and entropy bounds for concomitants of order statistics. Some comparisons are also made between the entropy of order statistics X _(r) and the entropy of its concomitants Y _[r]. In this family, we show that the mutual information between X _(r) and Y _[r], and Kullback–Leibler distance among the concomitants of order statistics are all distribution-free. Also, we compare the Pearson correlation coefficient between X _(r) and Y _[r] with the mutual information of (X _(r), Y _[r]) for the copula model of FGM family.     

Extensions to the modified Gram–Schmidt strategy and its application in the steepest ascent method

The orthogonalization of undesigned experiments is introduced to increase statistical precision of the estimated regression coefficients. The goals are to minimize the covariance and the bias of the least squares estimator for estimating the path of the steepest ascent (SA) that leads the users toward the neighbour of the optimum response. An orthogonal design is established to decrease the inverse determinant of X′X and the angle between the true and the estimated SA paths. For orthogonalization of an undesigned matrix, our proposed solution is constructed on the modified Gram–Schmidt strategy relevant to the process of Gaussian elimination. The proposed solution offers an orthogonal basis, in full working accuracy, for the space spanned by the columns of the original matrix.     

Estimation of P(Y < X) for the Three-Parameter Generalized Exponential Distribution

This article considers the estimation of R = P(Y < X) when X and Y are distributed as two independent three-parameter generalized exponential (GE) random variables with different shape parameters but having the same location and scale parameters. A modified maximum likelihood method and a Bayesian technique are used to estimate R on the basis of independent complete samples. The Bayes estimator cannot be obtained in explicit form, and therefore it has been determined using an importance sampling procedure. An analysis of a real life data set is presented for illustrative purposes.