Performance of Some Ridge Parameters for Probit Regression: With Application to Swedish Job Search Data

In ridge regression, the estimation of the ridge parameter is an important issue. This article generalizes some methods for estimating the ridge parameter for probit ridge regression (PRR) model based on the work of Kibria et al. (2011 Kibria, B. M. G., Månsson, K. and Shukur, G. 2011. Performance of some logistic ridge regression parameters. Computational Economics, DOI: 10.1007/s10614-011-9275-x [Google Scholar]). The performance of these new estimators is judged by calculating the mean squared error (MSE) using Monte Carlo simulations. In the design of the experiment, we chose to vary the sample size and the number of regressors. Furthermore, we generate explanatory variables that are linear combinations of other regressors, which is a common situation in economics. In an empirical application regarding Swedish job search data, we also illustrate the benefits of the new method.     

Jackknifing the Ridge Regression Estimator: A Revisit

Singh et al. (1986 Singh, B., Chaubey, Y.P., Dwivedi, T.D. (1986). An almost unbiased ridge estimator. Sankhya B48: 34236. [Google Scholar]) proposed an almost unbiased ridge estimator using Jackknife method that required transformation of the regression parameters. This article shows that the same method can be used to derive the Jackknifed ridge estimator of the original (untransformed) parameter without transformation. This method also leads in deriving easily the second-order Jackknifed ridge that may reduce the bias further. We further investigate the performance of these estimators along with a recent method by Batah et al. (2008 Batah, F. S.M., Ramanathan, T.V., Gore, S.D. (2008). The efficiency of modified Jack-knife and ridge type regression estimators: a comparison. Surv. Math. Applic. 3:111122. [Google Scholar]) called modified Jackknifed ridge theoretically as well as numerically.     

A Simulation Study of Some Ridge Regression Estimators under Different Distributional Assumptions

Based on the work of Khalaf and Shukur (2005 Khalaf , G. , Shukur , G. ( 2005 ). Choosing ridge parameters for regression problems . Communications in Statistics – Theory and Methods 34 : 1177 – 1182 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]), Alkhamisi et al. (2006 Alkhamisi , M. , Khalaf , G. , Shukur , G. ( 2006 ). Some modifications for choosing ridge parameters . Communications in Statistics – Theory and Methods 35 : 2005 – 2020 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]), and Muniz et al. (2010 Muniz , G. , Kibria , B. M. G. , Shukur , G. ( 2010 ). On developing ridge regression parameters: a graphical Investigation. Submitted for Publication . [Google Scholar]), this article considers several estimators for estimating the ridge parameter k. This article differs from aforementioned articles in three ways: (1) Data are generated from Normal, Student's t, and F distributions with appropriate degrees of freedom; (2) The number of regressors considered are from 4–12 instead of 2–4, which are the usual practice; (3) Both mean square error (MSE) and prediction sum of square (PRESS) are considered as the performance criterion. A simulation study has been conducted to compare the performance of the estimators. Based on the simulation study we found that, increasing the correlation between the independent variables has negative effect on the MSE and PRESS. However, increasing the number of regressors has positive effect on MSE and PRESS. When the sample size increases the MSE decreases even when the correlation between the independent variables is large. It is interesting to note that the dominance pictures of the estimators are remained the same under both the MSE and PRESS criterion. However, the performance of the estimators depends on the choice of the assumption of the error distribution of the regression model.     

Performance of some ridge regression estimators for the multinomial logit model

This article considers several estimators for estimating the ridge parameter k for multinomial logit model based on the work of Khalaf and Shukur (2005 Khalaf, G., and G. Shukur. 2005. Choosing ridge parameters for regression problems. Commun. Statist. Theor. Meth., 34:1177–1182.[Taylor &; Francis Online], [Web of Science ®] , [Google Scholar]), Alkhamisi et al. (2006 Alkhamisi, M., G. Khalaf, and G. Shukur. 2006. Some modifications for choosing ridge parameters. Commun. Statist. Theor. Meth. 35:2005–2020.[Taylor &; Francis Online], [Web of Science ®] , [Google Scholar]), and Muniz et al. (2012 Muniz, G., B. M. G. Kibria, K. Månsson, and G. Shukur. 2012. On developing ridge regression parameters: A graphical investigation. in SORT. 36: 115–138.[Web of Science ®] , [Google Scholar]). The mean square error (MSE) is considered as the performance criterion. A simulation study has been conducted to compare the performance of the estimators. Based on the simulation study we found that increasing the correlation between the independent variables and the number of regressors has negative effect on the MSE. However, when the sample size increases the MSE decreases even when the correlation between the independent variables is large. Based on the minimum MSE criterion some useful estimators for estimating the ridge parameter k are recommended for the practitioners.     

Some Modifications for Choosing Ridge Parameters

Standard least square regression can produce estimates having a large mean squares error (MSE) when predictor variables are highly correlated or multicollinear. In this article, we propose four modifications to choose the ridge parameter (K) when multicollinearity exists among the columns of the design matrix. The proposed new estimators are extended versions of that suggested by Khalaf and Shukur (2005 Khalaf , G. , Shukur , G. ( 2005 ). Choosing ridge parameter for regression problems . Commun. Statist. A 34 : 1177 – 1182 . [CSA] [Taylor & Francis Online] , [Google Scholar]). The properties of these estimators are compared with those of Hoerl and Kennard (1970a Hoerl , A. E. , Kennard , R. W. ( 1970a ). Ridge regression: biased estimation for non-orthogonal problems . Tech. . 12 : 55 – 67 . [CSA] [Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) and the OLS using the MSE criterion. All estimators under consideration are evaluated using simulation techniques under certain conditions where a number of factors that may affect their properties have been varied. In addition, it is shown that at least one of the proposed estimators either has a smaller MSE than the others or is the next best otherwise.     

A new estimator of finite population mean based on the dual use of the auxiliary information

When a sufficient correlation between the study variable and the auxiliary variable exists, the ranks of the auxiliary variable are also correlated with the study variable, and thus, these ranks can be used as an effective tool in increasing the precision of an estimator. In this paper, we propose a new improved estimator of the finite population mean that incorporates the supplementary information in forms of: (i) the auxiliary variable and (ii) ranks of the auxiliary variable. Mathematical expressions for the bias and the mean-squared error of the proposed estimator are derived under the first order of approximation. The theoretical and empirical studies reveal that the proposed estimator always performs better than the usual mean, ratio, product, exponential-ratio and -product, classical regression estimators, and Rao (1991 Rao, T.J. (1991). On certail methods of improving ration and regression estimators. Commun. Stat. Theory Methods 20(10):3325–3340.[Taylor &; Francis Online], [Web of Science ®] , [Google Scholar]), Singh et al. (2009 Singh, R., Chauhan, P., Sawan, N., Smarandache, F. (2009). Improvement in estimating the population mean using exponential estimator in simple random sampling. Int. J. Stat. Econ. 3(A09):13–18. [Google Scholar]), Shabbir and Gupta (2010 Shabbir, J., Gupta, S. (2010). On estimating finite population mean in simple and stratified random sampling. Commun. Stat. Theory Methods 40(2):199–212.[Taylor &; Francis Online], [Web of Science ®] , [Google Scholar]), Grover and Kaur (2011 Grover, L.K., Kaur, P. (2011). An improved estimator of the finite population mean in simple random sampling. Model Assisted Stat. Appl. 6(1):47–55. [Google Scholar], 2014) estimators.     

Combining Unbiased Ridge and Principal Component Regression Estimators

In the presence of multicollinearity problem, ordinary least squares (OLS) estimation is inadequate. To circumvent this problem, two well-known estimation procedures often suggested are the unbiased ridge regression (URR) estimator given by Crouse et al. (1995 Crouse , R. , Jin , C. , Hanumara , R. ( 1995 ). Unbiased ridge estimation with prior information and ridge trace . Commun. Statist. Theor. Meth. 24 : 2341 – 2354 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) and the (r, k) class estimator given by Baye and Parker (1984 Baye , M. , Parker , D. ( 1984 ). Combining ridge and principal component regression: a money demand illustration . Commun. Statist. Theor. Meth. 13 : 197 – 205 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]). In this article, we proposed a new class of estimators, namely modified (r, k) class ridge regression (MCRR) which includes the OLS, the URR, the (r, k) class, and the principal components regression (PCR) estimators. It is based on a criterion that combines the ideas underlying the URR and the PCR estimators. The standard properties of this new class estimator have been investigated and a numerical illustration is done. The conditions under which the MCRR estimator is better than the other two estimators have been investigated.     

Bayes minimax ridge regression estimators

The problem of estimating of the vector β of the linear regression model y = Aβ + ? with ? ～ N_p(0, σ²I_p) under quadratic loss function is considered when common variance σ² is unknown. We first find a class of minimax estimators for this problem which extends a class given by Maruyama and Strawderman (2005 Maruyama, Y., and W. E. Strawderman. 2005. A new class of generalized Bayes minimax ridge regression estimators. Annals of Statistics 33:1753–70.[Crossref], [Web of Science ®] , [Google Scholar]) and using these estimators, we obtain a large class of (proper and generalized) Bayes minimax estimators and show that the result of Maruyama and Strawderman (2005 Maruyama, Y., and W. E. Strawderman. 2005. A new class of generalized Bayes minimax ridge regression estimators. Annals of Statistics 33:1753–70.[Crossref], [Web of Science ®] , [Google Scholar]) is a special case of our result. We also show that under certain conditions, these generalized Bayes minimax estimators have greater numerical stability (i.e., smaller condition number) than the least-squares estimator.     

Sliced Average Variance Estimation for Censored Data

We consider a nonlinear censored regression problem with a vector of predictors. With censoring, high-dimensional regression analysis becomes much more complicated. Since censoring can cause severe bias in estimation, modification to adjust such bias is needed to be made. Based on the weight adjustment, we develop the modification of sliced average variance estimation for estimating the lifetime central subspace without requiring a prespecified parametric model. Our proposed method preserves as much regression information as possible. Simulation results are reported and comparisons are made with the sliced inverse regression of Li et al. (1999 Li , K. C. , Wang , J. L. , Chen , C. H. ( 1999 ). Dimension reduction for censored regression data . Ann. Statist. 27 : 1 – 23 . [Google Scholar]).     

Comment on a generalized stochastic restricted ridge regression estimator

In this note, we make some comments about the paper of Alheety and Kibria (2014 Alheety, M.I., Kibria, B.M.G. (2014). A generalized stochastic restricted ridge regression estimator. Commun. Stat. Theor. Meth. 43:4415–4427.[Taylor &; Francis Online], [Web of Science ®] , [Google Scholar]) and correct the wrongly proved Theorems in that paper.     

Bootstrap approach to test the homogeneity of order restricted mean vectors when the covariance matrices are unknown

Testing homogeneity of multivariate normal mean vectors under an order restriction when the covariance matrices are unknown, arbitrary positive definite and unequal are considered. This problem of testing has been studied to some extent, for example, by Kulatunga and Sasabuchi (1984 Kulatunga, D. D. S., Sasabuchi, S. (1984). A test of homogeneity of mean vectors against multivariate isotonic alternatives. Mem Fac Sci, Kyushu Univ Ser A Mathemat 38:151–161. [Google Scholar]) when the covariance matrices are known and also Sasabuchi et al. (2003 Sasabuchi, S., Tanaka, K., Tsukamodo, T. (2003). Testing homogeneity of multivariate normal mean vectors under an order restriction when the covariance matrices are common but unknown. Annals of Statistics. 31(5):1517–1536.[Web of Science ®] , [Google Scholar]) and Sasabuchi (2007 Sasabuchi, S. (2007). More powerful tests for homogeneity of multivariate normal mean vectors under an order restriction. Sankhya 69(4):700–716. [Google Scholar]) when the covariance matrices are unknown but common. In this paper, a test statistic is proposed and because of the main advantage of the bootstrap test is that it avoids the derivation of the complex null distribution analytically, a bootstrap test statistic is derived and since the proposed test statistic is location invariance the bootstrap p-value defined logical and some steps are presented to estimate it. Our numerical studies via Monte Carlo simulation show that the proposed bootstrap test can correctly control the type I error rates. The power of the test for some of the p-dimensional normal distributions is computed by Monte Carlo simulation. Also, the null distribution of test statistic is estimated using kernel density. Finally, the bootstrap test is illustrated using a real data.     

Estimation of finite population mean using two auxiliary variables in stratified two-phase sampling

We propose a new ratio type estimator for estimating the finite population mean using two auxiliary variables in stratified two-phase sampling. Expressions for bias and mean squared error of the proposed estimator are derived up to the first order of approximation. The proposed estimator is more efficient than the usual stratified sample mean estimator, traditional stratified ratio estimator and some other stratified estimators including Bahl and Tuteja (1991 Bahl, S., Tuteja, R. K. (1991). Ratio and product type exponential estimators. Information and Optimization Sciences 12:159–163. [Google Scholar]), Chami et al. (2012 Chami, P. S., Singh, B., Thomas, D. (2012). A two-prameter ratio-product-ratio estimator using auxiliary information. ISRN Probability and Statistics 2012:1–15, doi: 10.5402/2012/103860.[Crossref] , [Google Scholar]), Chand (1975 Chand, L. (1975) Some Ratio Type Estimator Based on two or more Auxiliary Variables, Ph.D. dissertation, Iowa State University, Ames, Iowa (unpublished). [Google Scholar]), Choudhury and Singh (2012 Choudhury, S., Singh, B. K. (2012). A class of chain ratio-product type estimators with two auxiliary variables under double sampling scheme. Journal of the Korean Statistical Society 41:247–256. [Google Scholar]), Hamad et al. (2013 Hamad, N., Hanif, M., Haider, N. (2013). A regression type estimator with two auxiliary variables for two-phase sampling. Open Journal of Statistics, 3:74–78. [Google Scholar]), Vishwakarma and Gangele (2014 Vishwakarma, G. K., Gangele, R. K. (2014). A class of chain ratio-type exponential estimators in double sampling using two auxiliary variates. Applied Mathematics and Computation 227:171–175. [Google Scholar]), Sanaullah et al. (2014 Sanaullah, A., Ali, H. M., Noor ul Amin, M., Hanif, M. (2014). Generalized exponential chain ratio estimators under stratified two-phase random sampling. Applied Mathematics and Computation 226:541–547. [Google Scholar]), and Chanu and Singh (2014 Chanu, W. K., Singh, B. K. (2014). Improved class of ratio-cum-product estimators of finite population mean in two phase sampling. Global Journal of Science Frontier Research: F Mathematics and Decision Sciences 14(2):1–14. [Google Scholar]).     

Modified Ridge Parameters for Seemingly Unrelated Regression Model

In this article, we modify a number of new biased estimators of seemingly unrelated regression (SUR) parameters which are developed by Alkhamisi and Shukur (2008 Alkhamisi , M. A. , Shukur , G. ( 2008 ). Developing ridge parameters for SUR model . Commun. Statist. Theor. Meth. 37 ( 4 ): 544 – 564 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]), AS, when the explanatory variables are affected by multicollinearity. Nine estimators of the ridge parameters have been modified and compared in terms of the trace mean squared error (TMSE) and (PR) criterion. The results from this extended study are the also compared with those founded by AS. A simulation study has been conducted to compare the performance of the modified estimators of the ridge parameters. The results showed that under certain conditions the performance of the multivariate ridge regression estimators based on SUR ridge R _MSmax is superior to other estimators in terms of TMSE and PR criterion. In large samples and when the collinearity between the explanatory variables is not high, the unbiased SUR, estimator produces a smaller TMSEs.     

A Log-Linear Regression Model for the Beta-Weibull Distribution

We introduce the log-beta Weibull regression model based on the beta Weibull distribution (Famoye et al., 2005 Famoye , F. , Lee , C. , Olumolade , O. ( 2005 ). The beta-Weibull distribution . Journal of Statistical Theory and Applications 4 : 121 – 136 . [Google Scholar]; Lee et al., 2007 Lee , C. , Famoye , F. , Olumolade , O. ( 2007 ). Beta-Weibull distribution: Some properties and applications to censored data . Journal of Modern Applied Statistical Methods 6 : 173 – 186 .[Crossref] , [Google Scholar]). We derive expansions for the moment generating function which do not depend on complicated functions. The new regression model represents a parametric family of models that includes as sub-models several widely known regression models that can be applied to censored survival data. We employ a frequentist analysis, a jackknife estimator, and a parametric bootstrap for the parameters of the proposed model. We derive the appropriate matrices for assessing local influences on the parameter estimates under different perturbation schemes and present some ways to assess global influences. Further, for different parameter settings, sample sizes, and censoring percentages, several simulations are performed. In addition, the empirical distribution of some modified residuals are displayed and compared with the standard normal distribution. These studies suggest that the residual analysis usually performed in normal linear regression models can be extended to a modified deviance residual in the proposed regression model applied to censored data. We define martingale and deviance residuals to evaluate the model assumptions. The extended regression model is very useful for the analysis of real data and could give more realistic fits than other special regression models.     

MSE Performance of a Heterogeneous Pre-Test Ridge Regression Estimator

This article describes how diagnostic procedures were derived for symmetrical nonlinear regression models, continuing the work carried out by Cysneiros and Vanegas (2008 Cysneiros , F. J. A. , Vanegas , L. H. ( 2008 ). Residuals and their statistical properties in symmetrical nonlinear models . Statist. Probab. Lett. 78 : 3269 – 3273 .[Crossref], [Web of Science ®] , [Google Scholar]) and Vanegas and Cysneiros (2010 Vanegas , L. H. , Cysneiros , F. J. A. ( 2010 ). Assesment of diagnostic procedures in symmetrical nonlinear regression models . Computat. Statist. Data Anal. 54 : 1002 – 1016 .[Crossref], [Web of Science ®] , [Google Scholar]), who showed that the parameters estimates in nonlinear models are more robust with heavy-tailed than with normal errors. In this article, we focus on assessing if the robustness of this kind of models is also observed in the inference process (i.e., partial F-test). Symmetrical nonlinear regression models includes all symmetric continuous distributions for errors covering both light- and heavy-tailed distributions such as Student-t, logistic-I and -II, power exponential, generalized Student-t, generalized logistic, and contaminated normal. Firstly, a statistical test is shown to evaluating the assumption that the error terms all have equal variance. The results of simulation studies which describe the behavior of the test for heteroscedasticity proposed in the presence of outliers are then given. To assess the robustness of inference process, we present the results of a simulation study which described the behavior of partial F-test in the presence of outliers. Also, some diagnostic procedures are derived to identify influential observations on the partial F-test. As ilustration, a dataset described in Venables and Ripley (2002 Venables , W. N. , Ripley , B. D. ( 2002 ). Modern Applied with S. , 4th ed. New York : Springer .[Crossref] , [Google Scholar]), is also analyzed.     

Some Classes of Estimators for the Population Median Using Auxiliary Information

This article considers some classes of estimators of the population median of the study variable using information on an auxiliary variable with their properties under large sample approximation. Asymptotic optimum estimator (AOE) in each class of estimators has been investigated along with the approximate mean square error formulae. It has been shown that the proposed classes of estimators are better than these considered by Gross (1980 Gross , T. S. ( 1980 ). Median estimation in sample surveys. Proc. Surv. Res. Meth. Sect. Amer. Statist. Assoc. 181–184 . [Google Scholar]), Kuk and Mak (1989 Kuk , A. Y. C. , Mak , T. K. ( 1989 ). Median estimation in the presence of auxiliary information . J. Roy. Statist. Soc. Ser. B51 : 261 – 269 . [Google Scholar]), Singh et al. (2003a Singh , H. P. , Singh , S. , Joarder , A. H. ( 2003a ). Estimation of population median when mode of an auxiliary variable is known . J. Statist. Res. 37 ( 1 ): 57 – 63 . [Google Scholar]), and Al and Cingi (2009 Al , S. , Cingi , H. ( 2009 ). New estimators for the population median in simple random sampling. Tenth Islamic Countries Conference on Statistical Sciences, held in New Cairo, Egypt . [Google Scholar]). An empirical study is carried out to judge the merits of the suggested class of estimators over other existing estimators.     

Empirical Likelihood Method for Censored Median Regression Models

Based on the semiparametric median regression analysis for the right-censored data developed by Ying et al. (1995 Ying , Z. , Jung , S. H. , Wei , L. J. ( 1995 ). Survival analysis with median regression models . J. Amer. Statist. Assoc. 90 : 178 – 184 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]), an empirical likelihood based inferential procedure for the regression coefficients is proposed. The limiting distribution of the proposed log-empirical likelihood ratio test statistic follows a chi-squared distribution, which corresponds to the standard asymptotic results of the empirical likelihood method. The inference about the subsets of the entire regression coefficients vector is discussed. The proposed method is illustrated by some simulation studies.     

A Monte Carlo Study of Recent Ridge Parameters

A number of procedures have been developed for finding biased estimators of regression parameters. One of these procedures is the ridge regression. In this article, a new approach to obtain the ridge parameter K is suggested and then evaluated by Monte Carlo simulations. A number of different models are investigated for different number of observations, the strength of correlation between the explanatory variables, and distribution of the error terms. The mean squared error (MSE) criterion is used to examine the performance of the proposed estimators when compared with other well-known estimators. Under certain conditions, it is shown that at least one of the proposed estimators have a smaller MSE than the ordinary least squared estimator (OLS) and Hoerl and Kennard (1970a Hoerl , A. E. , Kennard , R. W. ( 1970a ). Ridge regression: biased estimation for non-orthogonal problems . Technometrics 12 : 55 – 67 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]) estimator (HK).     

A Comparison of Procedures for Controlling the False Discovery Rate in the Presence of Small Variance Genes: A Simulation Study

The Significance Analysis of Microarrays (SAM; Tusher et al., 2001 Tusher , V. G. , Tibshirani , R. , Chu , G. ( 2001 ). Significance analysis of microarrys applied to the ionizing radiation response . Proceedings of the National Academy of Sciences 98 : 5116 – 5121 .[Crossref], [PubMed], [Web of Science ®] , [Google Scholar]) method is widely used in analyzing gene expression data while controlling the FDR by using resampling-based procedure in the microarray setting. One of the main components of the SAM procedure is the adjustment of the test statistic. The introduction of the fudge factor to the test statistic aims at deflating the large value of test statistics due to the small standard error of gene-expression. Lin et al. (2008 Lin , D. , Shkedy , Z. , Burzykowski , T. , Göhlmann , H. W. H. , De Bondt , A. , Perera , T. , Geerts , T. , Bijnens , L. ( 2008 ). Significance analysis of microarray (SAM) for comparisons of several treatments with one control . Biometric Journal, MCP 50 ( 5 ): 801 – 823 .[Crossref], [PubMed], [Web of Science ®] , [Google Scholar]) pointed out that the fudge factor does not effectively improve the power and the control of the FDR as compared to the SAM procedure without the fudge factor in the presence of small variance genes. Motivated by the simulation results presented in Lin et al. (2008 Lin , D. , Shkedy , Z. , Burzykowski , T. , Göhlmann , H. W. H. , De Bondt , A. , Perera , T. , Geerts , T. , Bijnens , L. ( 2008 ). Significance analysis of microarray (SAM) for comparisons of several treatments with one control . Biometric Journal, MCP 50 ( 5 ): 801 – 823 .[Crossref], [PubMed], [Web of Science ®] , [Google Scholar]), in this article, we extend our study to compare several methods for choosing the fudge factor in the modified t-type test statistics and use simulation studies to investigate the power and the control of the FDR of the considered methods.