Monotone fitting for developmental variables

In order to study developmental variables, for example, neuromotor development of children and adolescents, monotone fitting is typically needed. Most methods, to estimate a monotone regression function non-parametrically, however, are not straightforward to implement, a difficult issue being the choice of smoothing parameters. In this paper, a convenient implementation of the monotone B-spline estimates of Ramsay [Monotone regression splines in action (with discussion), Stat. Sci. 3 (1988), pp. 425–461] and Kelly and Rice [Montone smoothing with application to dose-response curves and the assessment of synergism, Biometrics 46 (1990), pp. 1071–1085] is proposed and applied to neuromotor data. Knots are selected adaptively using ideas found in Friedman and Silverman [Flexible parsimonous smoothing and additive modelling (with discussion), Technometrics 31 (1989), pp. 3–39] yielding a flexible algorithm to automatically and accurately estimate a monotone regression function. Using splines also simultaneously allows to include other aspects in the estimation problem, such as modeling a constant difference between two groups or a known jump in the regression function. Finally, an estimate which is not only monotone but also has a ‘levelling-off’ (i.e. becomes constant after some point) is derived. This is useful when the developmental variable is known to attain a maximum/minimum within the interval of observation.     

Performance of confidence interval tests for the ratio of two lognormal means applied to Weibull and gamma distribution data

Five estimation approaches have been developed to compute the confidence interval (CI) for the ratio of two lognormal means: (1) T, the CI based on the t-test procedure; (2) ML, a traditional maximum likelihood-based approach; (3) BT, a bootstrap approach; (4) R, the signed log-likelihood ratio statistic; and (5) R*, the modified signed log-likelihood ratio statistic. The purpose of this study was to assess the performance of these five approaches when applied to distributions other than lognormal distribution, for which they were derived. Performance was assessed in terms of average length and coverage probability of the CIs for each estimation approaches (i.e., T, ML, BT, R, and R*) when data followed a Weibull or gamma distribution. Four models were discussed in this study. In Model 1, the sample sizes and variances were equal within the two groups. In Model 2, the sample sizes were equal but variances were different within the two groups. In Model 3, the variances were different within the two groups and the larger variance was paired with the larger sample size. In Model 4, the variances were different within the two groups and the larger variance was paired with the smaller sample size. The results showed that when the variances of the two groups were equal, the t-test performed well, no matter what the underlying distribution was and how large the variances of the two groups were. The BT approach performed better than the others when the underlying distribution was not lognormal distribution, although it was inaccurate when the variances were large. The R* test did not perform well when the underlying distribution was Weibull or gamma distributed data, but it performed best when the data followed a lognormal distribution.     

Geometric characterization of planar regression

The geometric characterization of linear regression in terms of the ‘concentration ellipse’ by Galton [Galton, F., 1886, Family likeness in stature (with Appendix by Dickson, J.D.H.). Proceedings of the Royal Society of London, 40, 42–73.] and Pearson [Pearson, K., 1901, On lines and planes of closest fit to systems of points in space. Philosophical Magazine, 2, 559–572.] was extended to the case of unequal variances of the presumably uncorrelated errors in the experimental data [McCartin, B.J., 2003, A geometric characterization of linear regression. Statistics, 37(2), 101–117.]. In this paper, this geometric characterization is further extended to planar (and also linear) regression in three dimensions where a beautiful interpretation in terms of the concentration ellipsoid is developed.     

Bayes and Empirical Bayes Inference in Changepoint Problems

ABSTRACT

In the case of the random design nonparametric regression, the double smoothing technique is applied to estimate the multivariate regression function. The proposed estimator has desirable properties in both the finite sample and the asymptotic cases. In the finite sample case, it has bounded conditional (and unconditional) bias and variance. On the other hand, in the asymptotic case, it has the same mean square error as the local linear estimator in Fan (Design-Adaptive Nonparametric Regression. Journal of the American Statistical Association 1992, 87, 998–1004; Local Linear Regression Smoothers and Their Minimax Efficiencies. Annals of Statistics 1993, 21, 196–216). Simulation studies demonstrate that the proposed estimator is better than the local linear estimator, because it has a smaller sample mean integrated square error and gives smoother estimates.     

The performance of multivariate CUSUM control charts with estimated parameters

In this paper, we study the effect of estimating the vector of means and the variance–covariance matrix on the performance of two of the most widely used multivariate cumulative sum (CUSUM) control charts, the MCUSUM chart proposed by Crosier [Multivariate generalizations of cumulative sum quality-control schemes, Technometrics 30 (1988), pp. 291–303] and the MC1 chart proposed by Pignatiello and Runger [Comparisons of multivariate CUSUM charts, J. Qual. Technol. 22 (1990), pp. 173–186]. Using simulation, we investigate and compare the in-control and out-of-control performances of the competing charts in terms of the average run length measure. The in-control and out-of-control performances of the competing charts deteriorate significantly if the estimated parameters are used with control limits intended for known parameters, especially when only a few Phase I samples are used to estimate the parameters. We recommend the use of the MC1 chart over that of the MCUSUM chart if the parameters are estimated from a small number of Phase I samples.     

The sampling distribution of the W estimator of the number of valid signatures on a petition

Abstract

A new non linear estimator, W, for the number of valid, unique signatures on a petition has been shown better, for the cases enumerated and with certain restrictions, than a popular Goodman-type statistic, G. This article extends those results with relaxed conditions by developing the exact probability mass function and mean of W and a close approximation of the variance (Var(W)). If the proportion of valid signatures among unique and duplicated signatures is the same, then Var(W) is approximately a function of the means and variances of the two sample statistics. Using the delta method, we estimate Var(W), with the resulting approximation shown to be good, even when the condition of equal proportions does not hold. We compare W to G and establish which estimator is preferred for different intervals of the design parameters. Data from a Washington State petition illustrate the findings.     

A Note on Fitting a Regression without an Intercept Term

By entering the data (y _i ,x _i ) followed by (–y _i ,–x _i ), one can obtain an intercept-free regression Y = Xβ + ε from a program package that normally uses an intercept term. There is no bias in the resultant regression coefficients, but a minor postanalysis adjustment is needed to the residual variance and standard errors.     

Improving dispersion tests by symmetrization

Comparing the variances of several independent samples is a classic problem and many tests have been proposed in the literature. Conover et al. [Conover, W.J., Johnson, M.E. and Johnson, M.M., 1981, A comparative study of tests for homogeneity of variances with applications to the outer continental self bidding data. Technometrics, 23, 351–361.] and Shoemaker [Shoemaker, L.H., 1995, Tests for difference in dispersion based on quantiles. The American Statistician, 49 (2), 179–182.] find that the existing tests lack power for skewed sampling distributions. To address this problem, we studied the effect of an a priori symmetrization of the data on the performance of tests for homogeneity of variances. This article also updates the comprehensive comparative study of Conover et al.     

VARIANCE ESTIMATION IN THE ERROR COMPONENTS REGRESSION MODEL

ABSTRACT

In a regression model with a random individual and a random time effect explicit representations of the nonnegative quadratic minimum biased estimators of the corresponding variances are deduced. These estimators always exist and are unique. Moreover, under normality assumption of the dependent variable unbiased estimators of the mean squared errors of the variance estimates are derived. Finally, confidence intervals on the variance components are considered.     

Akaike Information Criterion for Selecting Variables in the Nested Error Regression Model

The Akaike Information Criterion (AIC) is developed for selecting the variables of the nested error regression model where an unobservable random effect is present. Using the idea of decomposing the likelihood into two parts of “within” and “between” analysis of variance, we derive the AIC when the number of groups is large and the ratio of the variances of the random effects and the random errors is an unknown parameter. The proposed AIC is compared, using simulation, with Mallows' C _p , Akaike's AIC, and Sugiura's exact AIC. Based on the rates of selecting the true model, it is shown that the proposed AIC performs better.     

Obtaining efficient estimates of park use and testing for the structural adequacy of models

This paper provides an examination of the problem of heteroscedasticity as it relates to estimating park use, although the results can also be applied to a wide variety of flow problems involving traffic, people or commodities. The major issue is that estimates of flows obtained using ordinary least squares, OLS, often yield statistically significant results while still giving rise to large differences between observed and predicted flows (residuals). The paper presents results which show that for the flow estimation problem of concern, more accurate use estimates may be obtained by using generalized least squares, GLS, rather than using OLS. Weights to use in GLS regression are developed taking into account the variance to be expected in origin-destination flows. It is shown that deriving the correct weights, estimates of variances, to use in a regression analysis results in an ‘absolute’ test for the structural appropriateness of the regression model. Tests related to the ‘absolute’ adequacy test are introduced and their use to identify specific structural problems with a model is illustrated.     

Testing for equivalence of variances using Hartley's ratio

Hartley's test for homogeneity of k normal‐distribution variances is based on the ratio between the maximum sample variance and the minimum sample variance. In this paper, the author uses the same statistic to test for equivalence of k variances. Equivalence is defined in terms of the ratio between the maximum and minimum population variances, and one concludes equivalence when Hartley's ratio is small. Exact critical values for this test are obtained by using an integral expression for the power function and some theoretical results about the power function. These exact critical values are available both when sample sizes are equal and when sample sizes are unequal. One related result in the paper is that Hartley's test for homogeneity of variances is no longer unbiased when the sample sizes are unequal. The Canadian Journal of Statistics 38: 647–664; 2010 © 2010 Statistical Society of Canada     

Bayesian estimation for time-series regressions improved with exact likelihoods

We propose an estimation procedure for time-series regression models under the Bayesian inference framework. With the exact method of Wise [Wise, J. (1955). The autocorrelation function and spectral density function. Biometrika, 42, 151–159], an exact likelihood function can be obtained instead of the likelihood conditional on initial observations. The constraints on the parameter space arising from the stationarity conditions are handled by a reparametrization, which was not taken into consideration by Chib [Chib, S. (1993). Bayes regression with autoregressive errors: A Gibbs sampling approach. J. Econometrics, 58, 275–294] or Chib and Greenberg [Chib, S. and Greenberg, E. (1994). Bayes inference in regression model with ARMA(p, q) errors. J. Econometrics, 64, 183–206]. Simulation studies show that our method leads to better inferential results than their results.     

Bayesian shrinkage estimates for regression coefficients in m populations

For a linear regression model over m populations with separate regression coefficients but a common error variance, a Bayesian model is employed to obtain regression coefficient estimates which are shrunk toward an overall value. The formulation uses Normal priors on the coefficients and diffuse priors on the grand mean vectors, the error variance, and the between-to-error variance ratios. The posterior density of the parameters which were given diffuse priors is obtained. From this the posterior means and variances of regression coefficients and the predictive mean and variance of a future observation are obtained directly by numerical integration in the balanced case, and with the aid of series expansions in the approximately balanced case. An example is presented and worked out for the case of one predictor variable. The method is an extension of Box & Tiao's Bayesian estimation of means in the balanced one-way random effects model.     

A Coefficient of Determination for Generalized Linear Models

The coefficient of determination, a.k.a. R², is well-defined in linear regression models, and measures the proportion of variation in the dependent variable explained by the predictors included in the model. To extend it for generalized linear models, we use the variance function to define the total variation of the dependent variable, as well as the remaining variation of the dependent variable after modeling the predictive effects of the independent variables. Unlike other definitions that demand complete specification of the likelihood function, our definition of R² only needs to know the mean and variance functions, so applicable to more general quasi-models. It is consistent with the classical measure of uncertainty using variance, and reduces to the classical definition of the coefficient of determination when linear regression models are considered.     

The Likelihood Ratio Test with the Box–Cox Transformation for the Normal Mixture Problem: Power and Sample Size Study

Abstract

Through simulation and regression, we study the alternative distribution of the likelihood ratio test in which the null hypothesis postulates that the data are from a normal distribution after a restricted Box–Cox transformation and the alternative hypothesis postulates that they are from a mixture of two normals after a restricted (possibly different) Box–Cox transformation. The number of observations in the sample is called N. The standardized distance between components (after transformation) is D = (μ₂ ? μ₁)/σ, where μ₁ and μ₂ are the component means and σ² is their common variance. One component contains the fraction π of observed, and the other 1 ? π. The simulation results demonstrate a dependence of power on the mixing proportion, with power decreasing as the mixing proportion differs from 0.5. The alternative distribution appears to be a non-central chi-squared with approximately 2.48 + 10N ^?0.75 degrees of freedom and non-centrality parameter 0.174N(D ? 1.4)² × [π(1 ? π)]. At least 900 observations are needed to have power 95% for a 5% test when D = 2. For fixed values of D, power, and significance level, substantially more observations are necessary when π ≥ 0.90 or π ≤ 0.10. We give the estimated powers for the alternatives studied and a table of sample sizes needed for 50%, 80%, 90%, and 95% power.     

Standard and robust orthogonal regression

A fast routine for converting regression algorithms into corresponding orthogonal regression (OR) algorithms was introduced in Ammann and Van Ness (1988). The present paper discusses the properties of various ordinary and robust OR procedures created using this routine. OR minimizes the sum of the orthogonal distances from the regression plane to the data points. OR has three types of applications. First, L ₂ OR is the maximum likelihood solution of the Gaussian errors-in-variables (EV) regression problem. This L ₂ solution is unstable, thus the robust OR algorithms created from robust regression algorithms should prove very useful. Secondly, OR is intimately related to principal components analysis. Therefore, the routine can also be used to create L ₁, robust, etc. principal components algorithms. Thirdly, OR treats the x and y variables symmetrically which is important in many modeling problems. Using Monte Carlo studies this paper compares the performance of standard regression, robust regression, OR, and robust OR on Gaussian EV data, contaminated Gaussian EV data, heavy-tailed EV data, and contaminated heavy-tailed EV data.     

Jackknife Estimation for Smooth Functions of the Parametric Component in Partially Linear Regression Models

Abstract

It is known that due to the existence of the nonparametric component, the usual estimators for the parametric component or its function in partially linear regression models are biased. Sometimes this bias is severe. To reduce the bias, we propose two jackknife estimators and compare them with the naive estimator. All three estimators are shown to be asymptotically equivalent and asymptotically normally distributed under some regularity conditions. However, through simulation we demonstrate that the jackknife estimators perform better than the naive estimator in terms of bias when the sample size is small to moderate. To make our results more useful, we also construct consistent estimators of the asymptotic variance, which are robust against heterogeneity of the error variances.     

Power-transformed linear regression on quantile residual life for censored competing risks data

ABSTRACT

This paper proposes a power-transformed linear quantile regression model for the residual lifetime of competing risks data. The proposed model can describe the association between any quantile of a time-to-event distribution among survivors beyond a specific time point and the covariates. Under covariate-dependent censoring, we develop an estimation procedure with two steps, including an unbiased monotone estimating equation for regression parameters and cumulative sum processes for the Box–Cox transformation parameter. The asymptotic properties of the estimators are also derived. We employ an efficient bootstrap method for the estimation of the variance–covariance matrix. The finite-sample performance of the proposed approaches are evaluated through simulation studies and a real example.     

Hypothesis testing in smoothing spline models

Nonparametric regression models are often used to check or suggest a parametric model. Several methods have been proposed to test the hypothesis of a parametric regression function against an alternative smoothing spline model. Some tests such as the locally most powerful (LMP) test by Cox et al. (Cox, D., Koh, E., Wahba, G. and Yandell, B. (1988). Testing the (parametric) null model hypothesis in (semiparametric) partial and generalized spline models. Ann. Stat., 16, 113–119.), the generalized maximum likelihood (GML) ratio test and the generalized cross validation (GCV) test by Wahba (Wahba, G. (1990). Spline models for observational data. CBMS-NSF Regional Conference Series in Applied Mathematics, SIAM.) were developed from the corresponding Bayesian models. Their frequentist properties have not been studied. We conduct simulations to evaluate and compare finite sample performances. Simulation results show that the performances of these tests depend on the shape of the true function. The LMP and GML tests are more powerful for low frequency functions while the GCV test is more powerful for high frequency functions. For all test statistics, distributions under the null hypothesis are complicated. Computationally intensive Monte Carlo methods can be used to calculate null distributions. We also propose approximations to these null distributions and evaluate their performances by simulations.