Model averaging procedure for partially linear single-index models

This paper is concerned with model selection and model averaging procedures for partially linear single-index models. The profile least squares procedure is employed to estimate regression coefficients for the full model and submodels. We show that the estimators for submodels are asymptotically normal. Based on the asymptotic distribution of the estimators, we derive the focused information criterion (FIC), formulate the frequentist model average (FMA) estimators and construct proper confidence intervals for FMA estimators and FIC estimator, a special case of FMA estimators. Monte Carlo studies are performed to demonstrate the superiority of the proposed method over the full model, and over models chosen by AIC or BIC in terms of coverage probability and mean squared error. Our approach is further applied to real data from a male fertility study to explore potential factors related to sperm concentration and estimate the relationship between sperm concentration and monobutyl phthalate.     

Model selection and model averaging for semiparametric partially linear models with missing data

We study model selection and model averaging in semiparametric partially linear models with missing responses. An imputation method is used to estimate the linear regression coefficients and the nonparametric function. We show that the corresponding estimators of the linear regression coefficients are asymptotically normal. Then a focused information criterion and frequentist model average estimators are proposed and their theoretical properties are established. Simulation studies are performed to demonstrate the superiority of the proposed methods over the existing strategies in terms of mean squared error and coverage probability. Finally, the approach is applied to a real data case.     

LASSO and shrinkage estimation in Weibull censored regression models

In this paper we address the problem of estimating a vector of regression parameters in the Weibull censored regression model. Our main objective is to provide natural adaptive estimators that significantly improve upon the classical procedures in the situation where some of the predictors may or may not be associated with the response. In the context of two competing Weibull censored regression models (full model and candidate submodel), we consider an adaptive shrinkage estimation strategy that shrinks the full model maximum likelihood estimate in the direction of the submodel maximum likelihood estimate. We develop the properties of these estimators using the notion of asymptotic distributional risk. The shrinkage estimators are shown to have higher efficiency than the classical estimators for a wide class of models. Further, we consider a LASSO type estimation strategy and compare the relative performance with the shrinkage estimators. Monte Carlo simulations reveal that when the true model is close to the candidate submodel, the shrinkage strategy performs better than the LASSO strategy when, and only when, there are many inactive predictors in the model. Shrinkage and LASSO strategies are applied to a real data set from Veteran's administration (VA) lung cancer study to illustrate the usefulness of the procedures in practice.     

Shrinkage averaging estimation

We discuss the impact of tuning parameter selection uncertainty in the context of shrinkage estimation and propose a methodology to account for problems arising from this issue: Transferring established concepts from model averaging to shrinkage estimation yields the concept of shrinkage averaging estimation (SAE) which reflects the idea of using weighted combinations of shrinkage estimators with different tuning parameters to improve overall stability, predictive performance and standard errors of shrinkage estimators. Two distinct approaches for an appropriate weight choice, both of which are inspired by concepts from the recent literature of model averaging, are presented: The first approach relates to an optimal weight choice with regard to the predictive performance of the final weighted estimator and its implementation can be realized via quadratic programming. The second approach has a fairly different motivation and considers the construction of weights via a resampling experiment. Focusing on Ridge, Lasso and Random Lasso estimators, the properties of the proposed shrinkage averaging estimators resulting from these strategies are explored by means of Monte-Carlo studies and are compared to traditional approaches where the tuning parameter is simply selected via cross validation criteria. The results show that the proposed SAE methodology can improve an estimators’ overall performance and reveal and incorporate tuning parameter uncertainty. As an illustration, selected methods are applied to some recent data from a study on leadership behavior in life science companies.     

Model averaging by jackknife criterion for varying-coefficient partially linear models

Abstract

This paper is concerned with model averaging procedure for varying-coefficient partially linear models. We proposed a jackknife model averaging method that involves minimizing a leave-one-out cross-validation criterion, and developed a computational shortcut to optimize the cross-validation criterion for weight choice. The resulting model average estimator is shown to be asymptotically optimal in terms of achieving the smallest possible squared error. The simulation studies have provided evidence of the superiority of the proposed procedures. Our approach is further applied to a real data.     

Lasso with convex loss: Model selection consistency and estimation

Abstract

Variable selection is a fundamental challenge in statistical learning if one works with data sets containing huge amount of predictors. In this artical we consider procedures popular in model selection: Lasso and adaptive Lasso. Our goal is to investigate properties of estimators based on minimization of Lasso-type penalized empirical risk with a convex loss function, in particular nondifferentiable. We obtain theorems concerning rate of convergence in estimation, consistency in model selection and oracle properties for Lasso estimators if the number of predictors is fixed, i.e. it does not depend on the sample size. Moreover, we study properties of Lasso and adaptive Lasso estimators on simulated and real data sets.     

Semiparametric multiple kernel estimators and model diagnostics for count regression functions

Abstract

This study concerns semiparametric approaches to estimate discrete multivariate count regression functions. The semiparametric approaches investigated consist of combining discrete multivariate nonparametric kernel and parametric estimations such that (i) a prior knowledge of the conditional distribution of model response may be incorporated and (ii) the bias of the traditional nonparametric kernel regression estimator of Nadaraya-Watson may be reduced. We are precisely interested in combination of the two estimations approaches with some asymptotic properties of the resulting estimators. Asymptotic normality results were showed for nonparametric correction terms of parametric start function of the estimators. The performance of discrete semiparametric multivariate kernel estimators studied is illustrated using simulations and real count data. In addition, diagnostic checks are performed to test the adequacy of the parametric start model to the true discrete regression model. Finally, using discrete semiparametric multivariate kernel estimators provides a bias reduction when the parametric multivariate regression model used as start regression function belongs to a neighborhood of the true regression model.     

On pre-test estimation of parametric functions in the general Gauss-Markov model with quadratic risk

The approach to preliminary test estimation based on comparing the weighted quadratic risk function of two competing estimators of β under the linear regression model {y,Xβ, σ² I} is extended to the case when a given vector of parametric functions κ=Kβ is to be estimated under the general Gauss-Markov model.     

Collective Labor Supply,Taxes, and Intrahousehold Allocation: An Empirical Approach

ABSTRACT

Most empirical studies of the impact of labor income taxation on the labor supply behavior of households use a unitary modeling approach. In this article, we empirically analyze income taxation and the choice of working hours by combining the collective approach for household behavior and the discrete hours choice framework with fixed costs of work. We identify the sharing rule parameters with data on working hours of both the husband and the wife within a couple. Parameter estimates are used to evaluate various model outcomes, like the wage elasticities of labor supply and the impacts of wage changes on the intrahousehold allocation of income. We also simulate the consequences of a policy change in the tax system. We find that the collective model has different empirical outcomes of income sharing than a restricted model that imposes income pooling. In particular, a specification with income pooling fails to capture asymmetries in the income sharing across spouses. These differences in outcomes have consequences for the evaluation of policy changes in the tax system and shed light on the effectiveness of certain policies.     

Generalized Least Squares Model Averaging

In this article, we propose a method of averaging generalized least squares estimators for linear regression models with heteroskedastic errors. The averaging weights are chosen to minimize Mallows’ C_p-like criterion. We show that the weight vector selected by our method is optimal. It is also shown that this optimality holds even when the variances of the error terms are estimated and the feasible generalized least squares estimators are averaged. The variances can be estimated parametrically or nonparametrically. Monte Carlo simulation results are encouraging. An empirical example illustrates that the proposed method is useful for predicting a measure of firms’ performance.     

Model averaging procedure for varying-coefficient partially linear models with missing responses

This paper is concerned with model averaging procedure for varying-coefficient partially linear models with missing responses. The profile least-squares estimation process and inverse probability weighted method are employed to estimate regression coefficients of the partially restricted models, in which the propensity score is estimated by the covariate balancing propensity score method. The estimators of the linear parameters are shown to be asymptotically normal. Then we develop the focused information criterion, formulate the frequentist model averaging estimators and construct the corresponding confidence intervals. Some simulation studies are conducted to examine the finite sample performance of the proposed methods. We find that the covariate balancing propensity score improves the performance of the inverse probability weighted estimator. We also demonstrate the superiority of the proposed model averaging estimators over those of existing strategies in terms of mean squared error and coverage probability. Finally, our approach is further applied to a real data example.     

Shrinkage ridge regression in partial linear models

ABSTRACT

In this paper, shrinkage ridge estimator and its positive part are defined for the regression coefficient vector in a partial linear model. The differencing approach is used to enjoy the ease of parameter estimation after removing the non parametric part of the model. The exact risk expressions in addition to biases are derived for the estimators under study and the region of optimality of each estimator is exactly determined. The performance of the estimators is evaluated by simulated as well as real data sets.     

A quasi-Bayesian model averaging approach for conditional quantile models

The value at risk (VaR) is a risk measure that is widely used by financial institutions to allocate risk. VaR forecast estimation involves the evaluation of conditional quantiles based on the currently available information. Recent advances in VaR evaluation incorporate conditional variance into the quantile estimation, which yields the conditional autoregressive VaR (CAViaR) models. However, uncertainty with regard to model selection in CAViaR model estimators raises the issue of identifying the better quantile predictor via averaging. In this study, we propose a quasi-Bayesian model averaging method that generates combinations of conditional VaR estimators based on single CAViaR models. This approach provides us a basis for comparing single CAViaR models against averaged ones for their ability to forecast VaR. We illustrate this method using simulated and financial daily return data series. The results demonstrate significant findings with regard to the use of averaged conditional VaR estimates when forecasting quantile risk.     

Compromise Pitman estimators

In this paper, we investigate the construction of compromise estimators of location and scale, by averaging over several models selected among a specified large set of possible models. The weight given to each distribution is based on the profile likelihood, which leads to a notion of distance between distributions as we study the asymptotic behaviour of such estimators. The selection of the models is made in a minimax way, in order to choose distributions that are close to any possible distribution. We also present simulation results of such compromise estimators based on contaminated Gaussian and Student's t distributions.     

The portfolio choice problem: comparison of certainty equivalence and optimal Bayes portfolios

For the portfolio problem with unknown parameter values, we compare the conventional certainty equivalence portfolio choice with the optimal Bayes portfolio. In the important single risky asset case a diffuse Bayes rule leads to portfolios that differ significantly from those suggested by a certainty equivalence rule which we show are inadmissible relative to a quadratic utility function for the range of parameters we consider. These results are invariant to arbitrary changes in the utility function parameters. We illustrate the results using a simple mutual fund example.     

Improved estimation of common location of two exponential populations with order restricted scale parameters using censored samples

ABSTRACT

Estimation of common location parameter of two exponential populations is considered when the scale parameters are ordered using type-II censored samples. A general inadmissibility result is proved which helps in deriving improved estimators. Further, a class of estimators dominating the MLE has been derived by an application of integrated expression of risk difference (IERD) approach of Kubokawa. A discussion regarding extending the results to a general k( ? 2) populations has been done. Finally, all the proposed estimators are compared through simulation.     

Estimating the error variance after a pre-test for an inequality restriction on the coefficients

Estimation of the regression error variance after a preliminary test of an inequality constraint on the coefficient vector is considered. We derive the exact finite sample risk of several inequality restricted and pre-test estimators of σ². These estimators are associated with the maximum likelihood, least squares and minimum mean squared error component estimators. Optimal critical values for the pre-test according to a mini-max regret criterion are numerically calculated. Furthermore, we examine the robustness of the optimal choice of critical values and the risk properties of the estimators of σ² to model mis-specification through the exclusion of relevant regressors.     

Flat-Top Realized Kernel Estimation of Quadratic Covariation With Nonsynchronous and Noisy Asset Prices

This article develops a general multivariate additive noise model for synchronized asset prices and provides a multivariate extension of the generalized flat-top realized kernel estimators, analyzed earlier by Varneskov (2014), to estimate its quadratic covariation. The additive noise model allows for α-mixing dependent exogenous noise, random sampling, and an endogenous noise component that encompasses synchronization errors, lead-lag relations, and diurnal heteroscedasticity. The various components may exhibit polynomially decaying autocovariances. In this setting, the class of estimators considered is consistent, asymptotically unbiased, and mixed Gaussian at the optimal rate of convergence, n^1/4. A simple finite sample correction based on projections of symmetric matrices ensures positive definiteness without altering the asymptotic properties of the estimators. It, thereby, guarantees the existence of nonlinear transformations of the estimated covariance matrix such as correlations and realized betas, which inherit the asymptotic properties from the flat-top realized kernel estimators. An empirically motivated simulation study assesses the choice of sampling scheme and projection rule, and it shows that flat-top realized kernels have a desirable combination of robustness and efficiency relative to competing estimators. Last, an empirical analysis of signal detection and out-of-sample predictions for a portfolio of six stocks of varying size and liquidity illustrates the use and properties of the new estimators.     

Time-consistent strategies for multi-period mean-variance portfolio optimization with the serially correlated returns

Abstract

In this paper, we discuss several different styles of multi-period mean-variance portfolio optimization problems under the serially correlated returns. We derive the time-consistent strategies for the classical multi-period mean-variance optimization with and without risk-free asset using a backward induction approach. We also propose an alternative multi-period mean-variance model, and the corresponding time-consistent strategies are derived. Whereafter, we provide some portfolio evaluation indexes and perform extensive empirical studies based on real data, aiming to provide useful advice for investors. To a large extent, the empirical results answer one important and practical question: in actual investment situations, which strategy is preferred by different investors?     

Marginal Likelihood Estimation with the Cross-Entropy Method

We consider an adaptive importance sampling approach to estimating the marginal likelihood, a quantity that is fundamental in Bayesian model comparison and Bayesian model averaging. This approach is motivated by the difficulty of obtaining an accurate estimate through existing algorithms that use Markov chain Monte Carlo (MCMC) draws, where the draws are typically costly to obtain and highly correlated in high-dimensional settings. In contrast, we use the cross-entropy (CE) method, a versatile adaptive Monte Carlo algorithm originally developed for rare-event simulation. The main advantage of the importance sampling approach is that random samples can be obtained from some convenient density with little additional costs. As we are generating independent draws instead of correlated MCMC draws, the increase in simulation effort is much smaller should one wish to reduce the numerical standard error of the estimator. Moreover, the importance density derived via the CE method is grounded in information theory, and therefore, is in a well-defined sense optimal. We demonstrate the utility of the proposed approach by two empirical applications involving women's labor market participation and U.S. macroeconomic time series. In both applications, the proposed CE method compares favorably to existing estimators.