Compound Regression and Constrained Regression: Nonparametric Regression Frameworks for EIV Models

Abstract

Errors-in-variable (EIV) regression is often used to gauge linear relationship between two variables both suffering from measurement and other errors, such as, the comparison of two measurement platforms (e.g., RNA sequencing vs. microarray). Scientists are often at a loss as to which EIV regression model to use for there are infinite many choices. We provide sound guidelines toward viable solutions to this dilemma by introducing two general nonparametric EIV regression frameworks: the compound regression and the constrained regression. It is shown that these approaches are equivalent to each other and, to the general parametric structural modeling approach. The advantages of these methods lie in their intuitive geometric representations, their distribution free nature, and their ability to offer candidate solutions with various optimal properties when the ratio of the error variances is unknown. Each includes the classic nonparametric regression methods of ordinary least squares, geometric mean regression (GMR), and orthogonal regression as special cases. Under these general frameworks, one can readily uncover some surprising optimal properties of the GMR, and truly comprehend the benefit of data normalization. Supplementary materials for this article are available online.     

Computer-Generated Neighbor Designs

Neighbor designs are useful to remove the neighbor effects. In this article, an algorithm is developed and is coded in Visual C + +to generate the initial block for possible first, second,…, and all order neighbor designs. To get the required design, a block (0, 1, 2,…, k ? 1) is then augmented with (v ? 1) blocks obtained by developing the initial block cyclically mod (v ? 1).     

Characterizing the BMAP/MAP/1 Departure Process via the ETAQA Truncation

Abstract

We propose a family of finite approximations for the departure process of a BMAP/MAP/1 queue. The departure process approximations are derived via an exact aggregate solution technique (called ETAQA) applied to M/G/1-type Markov processes. The proposed approximations are indexed by a parameter n(n > 1), which determines the size of the output model as n + 1 block levels of the M/G/1-type process. This output approximation preserves exactly the marginal distribution of the true departure process and the lag correlations of the interdeparture times up to lag n ? 2. Experimental results support the applicability of the proposed approximation in traffic-based decomposition of queueing networks.     

Inference for Adaptive Time Series Models: Stochastic Volatility and Conditionally Gaussian State Space Form

In this paper we model the Gaussian errors in the standard Gaussian linear state space model as stochastic volatility processes. We show that conventional MCMC algorithms for this class of models are ineffective, but that the problem can be alleviated by reparameterizing the model. Instead of sampling the unobserved variance series directly, we sample in the space of the disturbances, which proves to lower correlation in the sampler and thus increases the quality of the Markov chain.

Using our reparameterized MCMC sampler, it is possible to estimate an unobserved factor model for exchange rates between a group of n countries. The underlying n + 1 country-specific currency strength factors and the n + 1 currency volatility factors can be extracted using the new methodology. With the factors, a more detailed image of the events around the 1992 EMS crisis is obtained.

We assess the fit of competitive models on the panels of exchange rates with an effective particle filter and find that indeed the factor model is strongly preferred by the data.     

Nearly Gaussian Distributions and Application

We consider the random variable X that is not Gaussian but for which X ^c , where c = (2k + 1)/(2j + 1) with k, j ? {0, 1,…}, is approximately Gaussian. A variable of this type is used to model the errors made by meteorologists when forecasting temperatures.     

D-Optimal Designs for Covariate Parameters in Block Design Set Up

The problem considered is that of finding D-optimal design for the estimation of covariate parameters and the treatment and block contrasts in a block design set up in the presence of non stochastic controllable covariates, when N = 2(mod 4), N being the total number of observations. It is clear that when N ≠ 0 (mod 4), it is not possible to find designs attaining minimum variance for the estimated covariate parameters. Conditions for D-optimum designs for the estimation of covariate parameters were established when each of the covariates belongs to the interval [?1, 1]. Some constructions of D-optimal design have been provided for symmetric balanced incomplete block design (SBIBD) with parameters b = v, r = k = v ? 1, λ =v ? 2 when k = 2 (mod 4) and b is an odd integer.     

Bayesian Approach in Nonparametric Count Regression with Binomial Kernel

Recently, Kokonendji et al. have adapted the well-known Nadaraya–Watson kernel estimator for estimating the count function m in the context of nonparametric discrete regression. The authors have also investigated the bandwidth selection using the cross-validation method. In this article, we propose a Bayesian approach in the context of nonparametric count regression for estimating the bandwidth and the variance of the model error, which has not been estimated in Kokonendji et al. The model error is considered as Gaussian with mean of zero and a variance of σ². The Bayes estimates cannot be obtained in closed form and then, we use the well-known Markov chain Monte Carlo (MCMC) technique to compute the Bayes estimates under the squared errors loss function. The performance of this proposed approach and the cross-validation method are compared through simulation and real count data.     

A-Optimal and MV-Optimal Incomplete Block Designs for Comparing Successive Treatments

Consider an incomplete block experiment in which observations are taken from t treatments using an incomplete block design with b blocks of size k < t. Suppose the interest is in estimating the differences of effects of successive treatments. This may occur, for example, if the treatments are different dosages or concentrations of a compound. This article presents A-optimal and MV-optimal incomplete block designs for estimating the the differences of successive treatment effects. Tables of optimal designs are given for k < t ≤ 5 with b ≤ 40.     

Efficient Block Designs for Dependent Observations—A Computer-Aided Search

In this article, the exchange and interchange algorithm of Zergaw (1989 Zergaw , G. ( 1989 ). A sequential method of constructing optimal block designs . Austral. J. Statist. 31 : 333 – 342 .[Crossref] , [Google Scholar]) and Martin and Eccleston (1992 Martin , R. J. , Eccleston , J. A. ( 1992 ). Recursive formulae for constructing block designs with dependent errors . Biometrika 79 : 426 – 430 .[Crossref], [Web of Science ®] , [Google Scholar]) have been modified and used for searching efficient block designs for making all possible pairwise treatment comparisons when observations are dependent. The lower bounds to the A- and D-efficiencies of the designs in a given class of the designs have been obtained for correlated observation structure and the procedure of computing lower bounds to A- and D-efficiencies has been incorporated in the algorithm. The algorithm has been translated into a computer program using Microsoft Visual C++. Using this program, a search for efficient designs for making all possible pairwise treatment comparisons has been made for v ≤ 10, b ≤ 33, k ≤ 10 such that bk ≤ 100 and v > k. The block designs considered are usual block designs (rectangular block designs) and circular block designs. Nearest neighbor (NN), autoregressive of order 1 (AR(1)) correlation structures are studied. The ranges of correlation coefficients for different correlation structures investigated are |ρ|≤0.50 for NN correlation structure in rectangular blocks, |ρ|≤0.45 for NN correlation structure in circular blocks, and |ρ|≤0.95 for AR(1) correlation structure. For these ranges, the matrix of correlation coefficients among observations within a block is positive definite. Robustness aspects of designs that are efficient for a given value of correlation have been investigated against other values of correlation coefficients. Robustness aspects of designs that are efficient for independent observations have also been studied for experimental situations with dependent observations.     

Estimation and diagnostic for skew-normal partially linear models

Partially linear models (PLMs) are an important tool in modelling economic and biometric data and are considered as a flexible generalization of the linear model by including a nonparametric component of some covariate into the linear predictor. Usually, the error component is assumed to follow a normal distribution. However, the theory and application (through simulation or experimentation) often generate a great amount of data sets that are skewed. The objective of this paper is to extend the PLMs allowing the errors to follow a skew-normal distribution [A. Azzalini, A class of distributions which includes the normal ones, Scand. J. Statist. 12 (1985), pp. 171–178], increasing the flexibility of the model. In particular, we develop the expectation-maximization (EM) algorithm for linear regression models and diagnostic analysis via local influence as well as generalized leverage, following [H. Zhu and S. Lee, Local influence for incomplete-data models, J. R. Stat. Soc. Ser. B 63 (2001), pp. 111–126]. A simulation study is also conducted to evaluate the efficiency of the EM algorithm. Finally, a suitable transformation is applied in a data set on ragweed pollen concentration in order to fit PLMs under asymmetric distributions. An illustrative comparison is performed between normal and skew-normal errors.     

Cyclic Polygonal Designs with Block Size 3 and Joint Distance α = 2

Polygonal designs are useful in survey sampling in terms of balanced sampling plans excluding contiguous units (BSECs) and balanced sampling plans excluding adjacent units (BSAs). In this article, the method of cyclic shifts has been used for the construction of cyclic polygonal designs (in terms of BSAs) with block size k = 3 and λ = 1, 2, 3, 4, 6, 12 for joint distance α = 2 and 51 new designs for treatments v ≤ 100 are given.     

Inverse Regression in Binary Response LDV Model

Sliced inverse regression, a link-free and distribution-free method, is applied to binary response limited dependent variable models. An inverse regression property of binary response LDV model is found. Based on this property, if the distributions of X _j (j = 1, 2,…, p) satisfy the linearity condition, then β can be estimated up to a positive multiplicative scalar without any assumptions on the distribution of error ε. Moreover, the estimator can be proved to be asymptotically normal based on which testing hypotheses are considered. Simulations results are reported.     

Reply

ABSTRACT

In the class of stochastic volatility (SV) models, leverage effects are typically specified through the direct correlation between the innovations in both returns and volatility, resulting in the dynamic leverage (DL) model. Recently, two asymmetric SV models based on threshold effects have been proposed in the literature. As such models consider only the sign of the previous return and neglect its magnitude, this paper proposes a dynamic asymmetric leverage (DAL) model that accommodates the direct correlation as well as the sign and magnitude of the threshold effects. A special case of the DAL model with zero direct correlation between the innovations is the asymmetric leverage (AL) model. The dynamic asymmetric leverage models are estimated by the Monte Carlo likelihood (MCL) method. Monte Carlo experiments are presented to examine the finite sample properties of the estimator. For a sample size of T = 2000 with 500 replications, the sample means, standard deviations, and root mean squared errors of the MCL estimators indicate only a small finite sample bias. The empirical estimates for S&;P 500 and TOPIX financial returns, and USD/AUD and YEN/USD exchange rates, indicate that the DAL class, including the DL and AL models, is generally superior to threshold SV models with respect to AIC and BIC, with AL typically providing the best fit to the data.     

On-Line Process Control using Attributes with Misclassification Errors: An Economical Design for Short-Run Production

On-line process control consists of inspecting a single item for every m (integer and m ≥ 2) produced items. Based on the results of the inspection, it is decided whether the process is in-control (the fraction of conforming items is p ₁; State I) or out-of-control (the fraction of conforming items is p ₂ < p ₁; State II). If the inspected item is non conforming, it is determined that the process is out-of-control, and the production process is stopped for an adjustment; otherwise, production continues. As most designs of on-line process control assume a long-run production, this study can be viewed as an extension because it is concerned with short-run production and the decision regarding the process is subject to misclassification errors. The probabilistic model of the control system employs properties of an ergodic Markov chain to obtain the expression of the average cost of the system per unit produced, which can be minimised as a function of the sampling interval, m. The procedure is illustrated by a numerical example.     

Influence diagnostics for Student-t censored linear regression models

In this paper, we extend the censored linear regression model with normal errors to Student-t errors. A simple EM-type algorithm for iteratively computing maximum-likelihood estimates of the parameters is presented. To examine the performance of the proposed model, case-deletion and local influence techniques are developed to show its robust aspect against outlying and influential observations. This is done by the analysis of the sensitivity of the EM estimates under some usual perturbation schemes in the model or data and by inspecting some proposed diagnostic graphics. The efficacy of the method is verified through the analysis of simulated data sets and modelling a real data set first analysed under normal errors. The proposed algorithm and methods are implemented in the R package CensRegMod.     

Small-Sample Quantile Estimators in a Large Nonparametric Model

The large nonparametric model in this note is a statistical model with the family ? of all continuous and strictly increasing distribution functions. In the abundant literature of the subject, there are many proposals for nonparametric estimators that are applicable in the model. Typically the kth order statistic X _k:n is taken as a simplest estimator, with k = [nq], or k = [(n + 1)q], or k = [nq] + 1, etc. Often a linear combination of two consecutive order statistics is considered. In more sophisticated constructions, different L-statistics (e.g., Harrel–Davis, Kaigh–Lachenbruch, Bernstein, kernel estimators) are proposed. Asymptotically the estimators do not differ substantially, but if the sample size n is fixed, which is the case of our concern, differences may be serious. A unified treatment of quantile estimators in the large, nonparametric statistical model is developed.     

Robust Model-Based Inference for Incomplete Data via Penalized Spline Propensity Prediction

Parametric model-based regression imputation is commonly applied to missing-data problems, but is sensitive to misspecification of the imputation model. Little and An (2004 Little , R. J. A. , An , H. ( 2004 ). Robust likelihood-based analysis of multivariate data with missing values . Statistica Sinica 14 : 949 – 968 .[Web of Science ®] , [Google Scholar]) proposed a semiparametric approach called penalized spline propensity prediction (PSPP), where the variable with missing values is modeled by a penalized spline (P-Spline) of the response propensity score, which is logit of the estimated probability of being missing given the observed variables. Variables other than the response propensity are included parametrically in the imputation model. However they only considered point estimation based on single imputation with PSPP. We consider here three approaches to standard errors estimation incorporating the uncertainty due to non response: (a) standard errors based on the asymptotic variance of the PSPP estimator, ignoring sampling error in estimating the response propensity; (b) standard errors based on the bootstrap method; and (c) multiple imputation-based standard errors using draws from the joint posterior predictive distribution of missing values under the PSPP model. Simulation studies suggest that the bootstrap and multiple imputation approaches yield good inferences under a range of simulation conditions, with multiple imputation showing some evidence of closer to nominal confidence interval coverage when the sample size is small.     

Convergence Analysis of MCMC Algorithms for Bayesian Multivariate Linear Regression with Non‐Gaussian Errors

When Gaussian errors are inappropriate in a multivariate linear regression setting, it is often assumed that the errors are iid from a distribution that is a scale mixture of multivariate normals. Combining this robust regression model with a default prior on the unknown parameters results in a highly intractable posterior density. Fortunately, there is a simple data augmentation (DA) algorithm and a corresponding Haar PX‐DA algorithm that can be used to explore this posterior. This paper provides conditions (on the mixing density) for geometric ergodicity of the Markov chains underlying these Markov chain Monte Carlo algorithms. Letting d denote the dimension of the response, the main result shows that the DA and Haar PX‐DA Markov chains are geometrically ergodic whenever the mixing density is generalized inverse Gaussian, log‐normal, inverted Gamma (with shape parameter larger than d /2) or Fréchet (with shape parameter larger than d /2). The results also apply to certain subsets of the Gamma, F and Weibull families.     

Improved Score Tests in Symmetric Linear Regression Models

The class of symmetric linear regression models has the normal linear regression model as a special case and includes several models that assume that the errors follow a symmetric distribution with longer-than-normal tails. An important member of this class is the t linear regression model, which is commonly used as an alternative to the usual normal regression model when the data contain extreme or outlying observations. In this article, we develop second-order asymptotic theory for score tests in this class of models. We obtain Bartlett-corrected score statistics for testing hypotheses on the regression and the dispersion parameters. The corrected statistics have chi-squared distributions with errors of order O(n ^?3/2), n being the sample size. The corrections represent an improvement over the corresponding original Rao's score statistics, which are chi-squared distributed up to errors of order O(n ^?1). Simulation results show that the corrected score tests perform much better than their uncorrected counterparts in samples of small or moderate size.     

M-Estimators of Scale with Minimum Gross Errors Sensitivity

The median absolute deviation (MAD) is known to be the M-estimator of scale with minimum gross errors sensitivity (GES) when the error distribution is known to be symmetric and strongly unimodal. The problem considered here is to find the Fisher consistent M-estimator with minimum GES when the error distribution is symmetric but not necessarily unimodal. Under some general conditions, the score function χ corresponding to the minimizing M-estimator has the form χ(x) = ?1 when |x| < a; χ(x) = c when a < |x| < b; χ(x) = 1 when |x| > b. An example is given in which the M-estimator with minimum GES does not correspond to the MAD.