A variable selection approach to monotonic regression with Bernstein polynomials

One of the standard problems in statistics consists of determining the relationship between a response variable and a single predictor variable through a regression function. Background scientific knowledge is often available that suggests that the regression function should have a certain shape (e.g. monotonically increasing or concave) but not necessarily a specific parametric form. Bernstein polynomials have been used to impose certain shape restrictions on regression functions. The Bernstein polynomials are known to provide a smooth estimate over equidistant knots. Bernstein polynomials are used in this paper due to their ease of implementation, continuous differentiability, and theoretical properties. In this work, we demonstrate a connection between the monotonic regression problem and the variable selection problem in the linear model. We develop a Bayesian procedure for fitting the monotonic regression model by adapting currently available variable selection procedures. We demonstrate the effectiveness of our method through simulations and the analysis of real data.     

Orthogonal polynomials,repeated measures,and SPSS

We discuss the construction of discrete orthonormal polynomials, using MAPLE procedures. We also study two important applications of these polynomials in statistics: in multiple linear regression and in repeated measures analysis. In particular, it is argued that the tests given by SPSS for linear and other trends in a within-subject factor are inefficient. Examples are given, including two (from psychology and medicine, respectively) which involve repeated measures and SPSS. Extensive tables of discrete orthornormal polynomials are given in the Appendix.     

Weak Identification in Fuzzy Regression Discontinuity Designs

In fuzzy regression discontinuity (FRD) designs, the treatment effect is identified through a discontinuity in the conditional probability of treatment assignment. We show that when identification is weak (i.e., when the discontinuity is of a small magnitude), the usual t-test based on the FRD estimator and its standard error suffers from asymptotic size distortions as in a standard instrumental variables setting. This problem can be especially severe in the FRD setting since only observations close to the discontinuity are useful for estimating the treatment effect. To eliminate those size distortions, we propose a modified t-statistic that uses a null-restricted version of the standard error of the FRD estimator. Simple and asymptotically valid confidence sets for the treatment effect can be also constructed using this null-restricted standard error. An extension to testing for constancy of the regression discontinuity effect across covariates is also discussed. Supplementary materials for this article are available online.     

Change-Point Detection With Non-Parametric Regression

We consider the regression model y_i = ?(x_i ) + ε in which the function ? or its pth derivative ?^(p) may have a discontinuity at some unknown point τ. By fitting local polynomials from the left and right, we test the null that ?^(p) is continuous against the alternative that ?^(p)(τ?) ≠ ?^(p)(τ+). We obtain Darling-Erdös type limit theorems for the test statistics under the null hypothesis of no change, as well as their limits in probability under the alternative. Consistency of the related change-point estimators is also established.     

Change-Point Estimation in Long Memory Nonparametric Models with Applications

We consider the estimation of a change point or discontinuity in a regression function for random design model with long memory errors. We provide several change-point estimators and investigate the consistency of the estimators. Using the fractional ARIMA process as an example of long memory process, we report a small Monte Carlo experiment to compare the performance of the estimators in finite samples. We finish by applying the method to a climatological data example.     

M-Estimation in the partially linear model with Bernstein polynomials under shape constrains

We develope an M-estimator for partially linear models in which the nonparametric component is subject to various shape constraints. Bernstein polynomials are used to approximate the unknown nonparametric function, and shape constraints are imposed on the coefficients. Asymptotic normality of regression parameters and the optimal rate of convergence of the shape-restricted nonparametric function estimator are established under very mild conditions. Some simulation studies and a real data analysis are conducted to evaluate the finite sample performance of the proposed method.     

Optimal designs for statistical analysis with Zernike polynomials

The Zernike polynomials arise in several applications such as optical metrology or image analysis on a circular domain. In the present paper, we determine optimal designs for regression models which are represented by expansions in terms of Zernike polynomials. We consider two estimation methods for the coefficients in these models and determine the corresponding optimal designs. The first one is the classical least squares method and Φ _p -optimal designs in the sense of Kiefer [Kiefer, J., 1974, General equivalence theory for optimum designs (approximate theory). Annals of Statistics, 2 849–879.] are derived, which minimize an appropriate functional of the covariance matrix of the least squares estimator. It is demonstrated that optimal designs with respect to Kiefer's Φ _p -criteria (p>?∞) are essentially unique and concentrate observations on certain circles in the experimental domain. E-optimal designs have the same structure but it is shown in several examples that these optimal designs are not necessarily uniquely determined. The second method is based on the direct estimation of the Fourier coefficients in the expansion of the expected response in terms of Zernike polynomials and optimal designs minimizing the trace of the covariance matrix of the corresponding estimator are determined. The designs are also compared with the uniform designs on a grid, which is commonly used in this context.     

Understanding Estimators of Treatment Effects in Regression Discontinuity Designs

In this paper, we propose two new estimators of treatment effects in regression discontinuity designs. These estimators can aid understanding of the existing estimators such as the local polynomial estimator and the partially linear estimator. The first estimator is the partially polynomial estimator which extends the partially linear estimator by further incorporating derivative differences of the conditional mean of the outcome on the two sides of the discontinuity point. This estimator is related to the local polynomial estimator by a relocalization effect. Unlike the partially linear estimator, this estimator can achieve the optimal rate of convergence even under broader regularity conditions. The second estimator is an instrumental variable estimator in the fuzzy design. This estimator will reduce to the local polynomial estimator if higher order endogeneities are neglected. We study the asymptotic properties of these two estimators and conduct simulation studies to confirm the theoretical analysis.     

Detecting discontinuities in nonparametric regression curves and surfaces

The existence of a discontinuity in a regression function can be inferred by comparing regression estimates based on the data lying on different sides of a point of interest. This idea has been used in earlier research by Hall and Titterington (1992), Müller (1992) and later authors. The use of nonparametric regression allows this to be done without assuming linear or other parametric forms for the continuous part of the underlying regression function. The focus of the present paper is on assessing the evidence for the presence of a discontinuity within a regression function through examination of the standardised differences of ‘left’ and ‘right’ estimators at a variety of covariate values. The calculations for the test are carried out through distributional results on quadratic forms. A graphical method in the form of a reference band to highlight the sources of the evidence for discontinuities is proposed. The methods are also developed for the two covariate case where there are additional issues associated with the presence of a jump location curve. Methods for estimating this curve are also developed. All the techniques, for the one and two covariate situations, are illustrated through applications.     

Regression analysis of informative current status data with the semiparametric linear transformation model

Many methods have been developed in the literature for regression analysis of current status data with noninformative censoring and also some approaches have been proposed for semiparametric regression analysis of current status data with informative censoring. However, the existing approaches for the latter situation are mainly on specific models such as the proportional hazards model and the additive hazard model. Corresponding to this, in this paper, we consider a general class of semiparametric linear transformation models and develop a sieve maximum likelihood estimation approach for the inference. In the method, the copula model is employed to describe the informative censoring or relationship between the failure time of interest and the censoring time, and Bernstein polynomials are used to approximate the nonparametric functions involved. The asymptotic consistency and normality of the proposed estimators are established, and an extensive simulation study is conducted and indicates that the proposed approach works well for practical situations. In addition, an illustrative example is provided.     

Asymptotic results for parametric estimation in inadequate two phases regression models

We study the asymptotic properties, consistency, asymptotic normality, of the least squares estimator in a non linear regression problem. The model uses a parametric class Л of functions, but we do not assume that the unknown function belongs to that class. Л is here a class of continuous functions with a discontinuity in the first derivative. The problem of making a choice between two classes of that type is also studied.     

Regression Discontinuity Designs With Sample Selection

This article extends the standard regression discontinuity (RD) design to allow for sample selection or missing outcomes. We deal with both treatment endogeneity and sample selection. Identification in this article does not require any exclusion restrictions in the selection equation, nor does it require specifying any selection mechanism. The results can therefore be applied broadly, regardless of how sample selection is incurred. Identification instead relies on smoothness conditions. Smoothness conditions are empirically plausible, have readily testable implications, and are typically assumed even in the standard RD design. We first provide identification of the “extensive margin” and “intensive margin” effects. Then based on these identification results and principle stratification, sharp bounds are constructed for the treatment effects among the group of individuals that may be of particular policy interest, that is, those always participating compliers. These results are applied to evaluate the impacts of academic probation on college completion and final GPAs. Our analysis reveals striking gender differences at the extensive versus the intensive margin in response to this negative signal on performance.     

Including Covariates in the Regression Discontinuity Design

This article proposes a fully nonparametric kernel method to account for observed covariates in regression discontinuity designs (RDD), which may increase precision of treatment effect estimation. It is shown that conditioning on covariates reduces the asymptotic variance and allows estimating the treatment effect at the rate of one-dimensional nonparametric regression, irrespective of the dimension of the continuously distributed elements in the conditioning set. Furthermore, the proposed method may decrease bias and restore identification by controlling for discontinuities in the covariate distribution at the discontinuity threshold, provided that all relevant discontinuously distributed variables are controlled for. To illustrate the estimation approach and its properties, we provide a simulation study and an empirical application to an Austrian labor market reform. Supplementary materials for this article are available online.     

Gauss–Christoffel quadrature for inverse regression: applications to computer experiments

Sufficient dimension reduction (SDR) provides a framework for reducing the predictor space dimension in statistical regression problems. We consider SDR in the context of dimension reduction for deterministic functions of several variables such as those arising in computer experiments. In this context, SDR can reveal low-dimensional ridge structure in functions. Two algorithms for SDR—sliced inverse regression (SIR) and sliced average variance estimation (SAVE)—approximate matrices of integrals using a sliced mapping of the response. We interpret this sliced approach as a Riemann sum approximation of the particular integrals arising in each algorithm. We employ the well-known tools from numerical analysis—namely, multivariate numerical integration and orthogonal polynomials—to produce new algorithms that improve upon the Riemann sum-based numerical integration in SIR and SAVE. We call the new algorithms Lanczos–Stieltjes inverse regression (LSIR) and Lanczos–Stieltjes average variance estimation (LSAVE) due to their connection with Stieltjes’ method—and Lanczos’ related discretization—for generating a sequence of polynomials that are orthogonal with respect to a given measure. We show that this approach approximates the desired integrals, and we study the behavior of LSIR and LSAVE with two numerical examples. The quadrature-based LSIR and LSAVE eliminate the first-order algebraic convergence rate bottleneck resulting from the Riemann sum approximation, thus enabling high-order numerical approximations of the integrals when appropriate. Moreover, LSIR and LSAVE perform as well as the best-case SIR and SAVE implementations (e.g., adaptive partitioning of the response space) when low-order numerical integration methods (e.g., simple Monte Carlo) are used.

     

Discontinuities in robust nonparametric regression with α-mixing dependence

The main idea of the paper is to introduce a robust regression estimation method under an α-mixing dependence assumption, staying free of any parametric model restrictions while also allowing for some sudden changes in the unknown regression function. The sudden changes in the model may correspond to discontinuity points (jumps) or higher order breaks (jumps in corresponding derivatives) as well. We firstly derive some important statistical properties for local polynomial M-smoother estimates and we will propose a statistical test to decide whether some given point of interest is significantly important for a change to occur or not. As the asymptotic distribution of the test statistic depends on quantities which are left unknown we also introduce a bootstrap algorithm which can be used to mimic the target distribution of interest. All necessary proofs are provided together with some experimental results from a simulation study and a real data example.     

Beyond Incentives: Do Schools Use Accountability Rewards Productively?

We use a regression discontinuity design to analyze an understudied aspect of school accountability systems—how schools use financial rewards. For two years, California's accountability system financially rewarded schools based on a deterministic function of test scores. Qualifying schools received per-pupil awards amounting to about 1% of statewide per-pupil spending. Corroborating anecdotal evidence that awards were paid out as teacher bonuses, we find no evidence that winning schools purchased more instructional material, increased teacher hiring, or changed the subject-specific composition of their teaching staff. Most importantly, we find no evidence that student achievement increased in winning schools. Supplemental materials for this article are available online.     

Estimation and inference in regression discontinuity designs with asymmetric kernels

We study the behaviour of the Wald estimator of causal effects in regression discontinuity design when local linear regression (LLR) methods are combined with an asymmetric gamma kernel. We show that the resulting statistic is no more complex to implement than existing methods, remains consistent at the usual non-parametric rate, and maintains an asymptotic normal distribution but, crucially, has bias and variance that do not depend on kernel-related constants. As a result, the new estimator is more efficient and yields more reliable inference. A limited Monte Carlo experiment is used to illustrate the efficiency gains. As a by product of the main discussion, we extend previous published work by establishing the asymptotic normality of the LLR estimator with a gamma kernel. Finally, the new method is used in a substantive application.     

The Use of Polynomials to Shift Coefficients in Linear Regression Models

Polynomials are commonly used in linear regression models to capture nonlinearities in explanatory variables. It is less common, however, that polynomials are used to shift the regression coefficients, an exception being the use of polynomially distributed lag coefficients. This note recommends the technique for a wider range of applications and suggests the Lagrangian interpolation representation as the most convenient for practitioners.     

Minimum Contrast Empirical Likelihood Inference of Discontinuity in Density*

Abstract

This article investigates the asymptotic properties of a simple empirical-likelihood-based inference method for discontinuity in density. The parameter of interest is a function of two one-sided limits of the probability density function at (possibly) two cut-off points. Our approach is based on the first-order conditions from a minimum contrast problem. We investigate both first-order and second-order properties of the proposed method. We characterize the leading coverage error of our inference method and propose a coverage-error-optimal (CE-optimal, hereafter) bandwidth selector. We show that the empirical likelihood ratio statistic is Bartlett correctable. An important special case is the manipulation testing problem in a regression discontinuity design (RDD), where the parameter of interest is the density difference at a known threshold. In RDD, the continuity of the density of the assignment variable at the threshold is considered as a “no-manipulation” behavioral assumption, which is a testable implication of an identifying condition for the local average treatment effect. When specialized to the manipulation testing problem, the CE-optimal bandwidth selector has an explicit form. We propose a data-driven CE-optimal bandwidth selector for use in practice. Results from Monte Carlo simulations are presented. Usefulness of our method is illustrated by an empirical example.     

Nonparametric kernel estimation of the impact of tax policy on the demand for private health insurance in Australia

This paper is motivated by our attempt to answer a policy question: how is private health insurance take‐up in Australia affected by the income threshold at which the Medicare Levy Surcharge (MLS) kicks in? We propose a new difference deconvolution kernel estimator for the location and size of regression discontinuities. We also propose a bootstrapping procedure for estimating the confidence interval for the estimated discontinuity. Performance of the estimator is evaluated by Monte Carlo simulations before it is applied to estimating the effect of the income threshold of MLS on the take‐up of private health insurance in Australia, using contaminated data.