Regression Methods for Combining Multiple Classifiers

As no single classification method outperforms other classification methods under all circumstances, decision-makers may solve a classification problem using several classification methods and examine their performance for classification purposes in the learning set. Based on this performance, better classification methods might be adopted and poor methods might be avoided. However, which single classification method is the best to predict the classification of new observations is still not clear, especially when some methods offer similar classification performance in the learning set. In this article we present various regression and classical methods, which combine several classification methods to predict the classification of new observations. The quality of the combined classifiers is examined on some real data. Nonparametric regression is the best method of combining classifiers.     

Eliciting Subjective Survival Curves: Lessons from Partial Identification

When analyzing data on subjective expectations of continuous outcomes, researchers have access to a limited number of reported probabilities for each respondent from which to construct complete distribution functions. Moreover, reported probabilities may be rounded and thus not equal to true beliefs. Using survival expectations elicited from a representative sample from the Netherlands, we investigate what can be learned if we take these two sources of missing information into account and expectations are therefore only partially identified. We find novel evidence for rounding by checking whether reported expectations are consistent with a hazard of death that increases weakly with age. Only 39% of reported beliefs are consistent with this under the assumption that all probabilities are reported precisely, while 92% are if we allow for rounding. Using the available information to construct bounds on subjective life expectancy, we show that the data alone are not sufficiently informative to allow for useful inference in partially identified linear models, even in the absence of rounding. We propose to improve precision by interpolation between rounded probabilities. Interpolation in combination with a limited amount of rounding does yield informative intervals.     

Tail Behavior and Limit Distribution of Maximum of Logarithmic General Error Distribution

Logarithmic general error distribution, an extension of the log-normal distribution, is proposed. Some interesting properties of the log GED are derived. These properties are applied to establish the asymptotic behavior of the ratio of probability densities and the ratio of the tails of the logarithmic general error and log-normal distributions, and to derive the asymptotic distribution of the partial maximum of an independent and identically distributed sequence obeying the log GED.     

To be or not to be valid in testing the significance of the slope in simple quantitative linear models with autocorrelated errors

In this article, the validity of procedures for testing the significance of the slope in quantitative linear models with one explanatory variable and first-order autoregressive [AR(1)] errors is analyzed in a Monte Carlo study conducted in the time domain. Two cases are considered for the regressor: fixed and trended versus random and AR(1). In addition to the classical t -test using the Ordinary Least Squares (OLS) estimator of the slope and its standard error, we consider seven t -tests with n-2,hbox{df} built on the Generalized Least Squares (GLS) estimator or an estimated GLS estimator, three variants of the classical t -test with different variances of the OLS estimator, two asymptotic tests built on the Maximum Likelihood (ML) estimator, the F -test for fixed effects based on the Restricted Maximum Likelihood (REML) estimator in the mixed-model approach, two t -tests with n - 2 df based on first differences (FD) and first-difference ratios (FDR), and four modified t -tests using various corrections of the number of degrees of freedom. The FDR t -test, the REML F -test and the modified t -test using Dutilleul's effective sample size are the most valid among the testing procedures that do not assume the complete knowledge of the covariance matrix of the errors. However, modified t -tests are not applicable and the FDR t -test suffers from a lack of power when the regressor is fixed and trended ( i.e. , FDR is the same as FD in this case when observations are equally spaced), whereas the REML algorithm fails to converge at small sample sizes. The classical t -test is valid when the regressor is fixed and trended and autocorrelation among errors is predominantly negative, and when the regressor is random and AR(1), like the errors, and autocorrelation is moderately negative or positive. We discuss the results graphically, in terms of the circularity condition defined in repeated measures ANOVA and of the effective sample size used in correlation analysis with autocorrelated sample data. An example with environmental data is presented.     

LS estimation of periodic autoregressive models with non-Gaussian errors: a simulation study

In this article, the least squares (LS) estimates of the parameters of periodic autoregressive (PAR) models are investigated for various distributions of error terms via Monte-Carlo simulation. Beside the Gaussian distribution, this study covers the exponential, gamma, student-t, and Cauchy distributions. The estimates are compared for various distributions via bias and MSE criterion. The effect of other factors are also examined as the non-constancy of model orders, the non-constancy of the variances of seasonal white noise, the period length, and the length of the time series. The simulation results indicate that this method is in general robust for the estimation of AR parameters with respect to the distribution of error terms and other factors. However, the estimates of those parameters were, in some cases, noticeably poor for Cauchy distribution. It is also noticed that the variances of estimates of white noise variances are highly affected by the degree of skewness of the distribution of error terms.     

On the accuracy and cost of numerical integration in several variables ∗

Identical numerical integration experiments are performed on a CYBER 205 and an IBM 3081 in order to gauge the relative performance of several methods of integration. The methods employed are the general methods of Gauss-Legendre, iterated Gauss-Legendre, Newton-Cotes, Romberg and Monte Carlo as well as three methods, due to Owen, Dutt, and Clark respectively, for integrating the normal density. The bi- and trivariate normal densities and four other functions are integrated; the latter four have integrals expressible in closed form and some of them can be parameterized to exhibit singularities or highly periodic behavior. The various Gauss-Legendre methods tend to be most accurate (when applied to the normal density they are even more accurate than the special purpose methods designed for the normal) and while they are not the fastest, they are at least competitive. In scalar mode the CYBER is about 2-6 times faster than the IBM 3081 and the speed advantage of vectorised to scalar mode ranges from 6 to 15. Large scale econometric problems of the probit type should now be routinely soluble.     

The peculiar shrinkage properties of partial least squares regression

Partial least squares regression has been widely adopted within some areas as a useful alternative to ordinary least squares regression in the manner of other shrinkage methods such as principal components regression and ridge regression. In this paper we examine the nature of this shrinkage and demonstrate that partial least squares regression exhibits some undesirable properties.     

Local polynomial estimation of Poisson intensities in the presence of reporting delays

Summary.  The system for monitoring suicides in Hong Kong has considerable delays in reporting as the cause of death needs to be determined by a coroner's investigation. However, timely estimates of suicide rates are desirable to assist in the formulation of public health policies. This motivated us to develop a non-parametric procedure to estimate the intensity function of a Poisson process in the presence of reporting delays. We give closed form estimators of the Poisson intensity and the delay distribution, conduct simulation studies to evaluate the method proposed and derive their asymptotic properties. The method proposed is applied to estimate the intensity of suicide in Hong Kong.     

Study of partial least squares and ridge regression methods

This article considers both Partial Least Squares (PLS) and Ridge Regression (RR) methods to combat multicollinearity problem. A simulation study has been conducted to compare their performances with respect to Ordinary Least Squares (OLS). With varying degrees of multicollinearity, it is found that both, PLS and RR, estimators produce significant reductions in the Mean Square Error (MSE) and Prediction Mean Square Error (PMSE) over OLS. However, from the simulation study it is evident that the RR performs better when the error variance is large and the PLS estimator achieves its best results when the model includes more variables. However, the advantage of the ridge regression method over PLS is that it can provide the 95% confidence interval for the regression coefficients while PLS cannot.