A Method for Analyzing Supersaturated Designs

ABSTRACT

Analyzing supersaturated designs is challenging because the number of experiments is less than the number of factors. In this article we propose a new contrasts based method to analyze supersaturated designs. The method is discussed and explained through some simulation examples. The performance of the method is evaluated using several known designs from the literature.     

A Method for Analyzing Supersaturated Designs with a Block Orthogonal Structure

Supersaturated designs is a large class of factorial designs which can be used for screening out the important factors from a large set of potentially active variables. The huge advantage of these designs is that they reduce the experimental cost drastically, but their critical disadvantage is the confounding involved in the statistical analysis. In this article, we propose a method for analyzing data using a specific type of supersaturated designs. This method heavily uses the special block orthogonal structure of the supersaturated designs given by Tang and Wu (1997 Tang , B. , Wu , C. F. J. ( 1997 ). A method for constructing supersaturated designs and its Es ²-optimality . Canadian J. Statist. 25 : 191 – 201 .[Crossref], [Web of Science ®] , [Google Scholar]). Also, we compare our method with several known statistical analysis methods by using some of the existing supersaturated designs. The comparison is performed by some simulating experiments and the Type I and Type II error rates are calculated. The results are presented in tables and the discussion to follow.     

A three-stage variable selection method for supersaturated designs

A supersaturated design (SSD) is a design whose run size is not enough for estimating all main effects. Such a design is commonly used in screening experiments to screen active effects based on the effect sparsity principle. Traditional approaches, such as the ordinary stepwise regression and the best subset variable selection, may not be appropriate in this situation. In this article, a new variable selection method is proposed based on the idea of staged dimensionality reduction. Simulations and several real data studies indicate that the newly proposed method is more effective than the existing data analysis methods.     

A New Method for the Analysis of Supersaturated Designs with Discrete Data

Supersaturated designs are factorial designs in which the number of potential effects is greater than the run size. They are commonly used in screening experiments, with the aim of identifying the dominant active factors with low cost. However, an important research field, which is poorly developed, is the analysis of such designs with non-normal response. In this article, we develop a variable selection strategy, through the modification of the PageRank algorithm, which is commonly used in the Google search engine for ranking Webpages. The proposed method incorporates an appropriate information theoretical measure into this algorithm and as a result, it can be efficiently used for factor screening. A noteworthy advantage of this procedure is that it allows the use of supersaturated designs for analyzing discrete data and therefore a generalized linear model is assumed. As it is depicted via a thorough simulation study, in which the Type I and Type II error rates are computed for a wide range of underlying models and designs, the presented approach can be considered quite advantageous and effective.     

Construction of Efficient Multi-Level k-Circulant Supersaturated Designs

This article proposes an algorithm to construct efficient balanced multi-level k-circulant supersaturated designs with m factors and n runs. The algorithm generates efficient balanced multi-level k-circulant supersaturated designs very fast. Using the proposed algorithm many balanced multi-level supersaturated designs are constructed and cataloged. A list of many optimal and near optimal, multi-level supersaturated designs is also provided for m ≤ 60 and number of levels (q) ≤10. The algorithm can be used to generate two-level k-circulant supersaturated designs also and some large optimal two-level supersaturated designs are presented. An upper bound to the number of factors in a balanced multi-level supersaturated design such that no two columns are fully aliased is also provided.     

Analyzing Supersaturated Designs by Means of an Information Based Criterion

The cost and time consumption of many industrial experimentations can be reduced using the class of supersaturated designs since this can be used for screening out the important factors from a large set of potentially active variables. A supersaturated design is a design for which there are fewer runs than effects to be estimated. Although there exists a wide study of construction methods for supersaturated designs, their analysis methods are yet in an early research stage. In this article, we propose a method for analyzing data using a correlation-based measure, named as symmetrical uncertainty. This method combines measures from the information theory field and is used as the main idea of variable selection algorithms developed in data mining. In this work, the symmetrical uncertainty is used from another viewpoint in order to determine more directly the important factors. The specific method enables us to use supersaturated designs for analyzing data of generalized linear models for a Bernoulli response. We evaluate our method by using some of the existing supersaturated designs, obtained according to methods proposed by Tang and Wu (1997 Tang , B. , Wu , C. F. J. (1997). A method for constructing supersaturated designs and its E(s ²)-optimality. Canadian Journal of Statistics 25:191–201.[Crossref], [Web of Science ®] , [Google Scholar]) as well as by Koukouvinos et al. (2008 Koukouvinos , C. , Mylona , K. , Simos , D. E. ( 2008 ). E(s ²)-optimal and minimax-optimal cyclic supersaturated designs via multi-objective simulated annealing . Journal of Statistical Planning and Inference 138 : 1639 – 1646 .[Crossref], [Web of Science ®] , [Google Scholar]). The comparison is performed by some simulating experiments and the Type I and Type II error rates are calculated. Additionally, Receiver Operating Characteristics (ROC) curves methodology is applied as an additional statistical tool for performance evaluation.     

Robust Variable Selection in Linear Mixed Models

In this article, we develop a robust variable selection procedure jointly for fixed and random effects in linear mixed models for longitudinal data. We propose a penalized robust estimator for both the regression coefficients and the variance of random effects based on a re-parametrization of the linear mixed models. Under some regularity conditions, we show the oracle properties of the proposed robust variable selection method. Simulation study shows the robustness of the proposed method against outliers. In the end, the proposed methods is illustrated in the analysis of a real data set.     

Variable selection in linear regression analysis with alternative Bayesian information criteria using differential evaluation algorithm

In statistical analysis, one of the most important subjects is to select relevant exploratory variables that perfectly explain the dependent variable. Variable selection methods are usually performed within regression analysis. Variable selection is implemented so as to minimize the information criteria (IC) in regression models. Information criteria directly affect the power of prediction and the estimation of selected models. There are numerous information criteria in literature such as Akaike Information Criteria (AIC) and Bayesian Information Criteria (BIC). These criteria are modified for to improve the performance of the selected models. BIC is extended with alternative modifications towards the usage of prior and information matrix. Information matrix-based BIC (IBIC) and scaled unit information prior BIC (SPBIC) are efficient criteria for this modification. In this article, we proposed a combination to perform variable selection via differential evolution (DE) algorithm for minimizing IBIC and SPBIC in linear regression analysis. We concluded that these alternative criteria are very useful for variable selection. We also illustrated the efficiency of this combination with various simulation and application studies.     

Variable Selection for Support Vector Machines

Consider using values of variables X ₁, X ₂,…, X _p to classify entities into one of two classes. Kernel-based procedures such as support vector machines (SVMs) are well suited for this task. In general, the classification accuracy of SVMs can be substantially improved if instead of all p candidate variables, a smaller subset of (say m) variables is used. A new two-step approach to variable selection for SVMs is therefore proposed: best variable subsets of size k = 1,2,…, p are first identified, and then a new data-dependent criterion is used to determine a value for m. The new approach is evaluated in a Monte Carlo simulation study, and on a sample of data sets.     

Variable Selection in Semiparametric Quantile Modeling for Longitudinal Data

We propose a penalized quantile regression for partially linear varying coefficient (VC) model with longitudinal data to select relevant non parametric and parametric components simultaneously. Selection consistency and oracle property are established. Furthermore, if linear part and VC part are unknown, we propose a new unified method, which can do three types of selections: separation of varying and constant effects, selection of relevant variables, and it can be carried out conveniently in one step. Consistency in the three types of selections and oracle property in estimation are established as well. Simulation studies and real data analysis also confirm our method.     

Variable Selection for Semiparametric Partially Linear Covariate-Adjusted Regression Models

In this article, the partially linear covariate-adjusted regression models are considered, and the penalized least-squares procedure is proposed to simultaneously select variables and estimate the parametric components. The rate of convergence and the asymptotic normality of the resulting estimators are established under some regularization conditions. With the proper choices of the penalty functions and tuning parameters, it is shown that the proposed procedure can be as efficient as the oracle estimators. Some Monte Carlo simulation studies and a real data application are carried out to assess the finite sample performances for the proposed method.     

Input Variable Selection in Neural Network Models

One of the most important issues in using neural networks for the analysis of real-world problems is the input variable selection problem. This article connects input variable selection with multiple testing in the neural network regression models. In the proposed procedure, the number and the type of input neurons are selected by means of a testing scheme, based on appropriate measures of relevance of a given input variable to the model. In order to avoid the data snooping problem, family-wise error rate is controlled by using the StepM method proposed by Romano and Wolf (2005 Romano , J. P. , Wolf , M. ( 2005 ). Exact and approximate stepdown methods for multiple hypothesis testing . J. Amer. Statist. Assoc. 100 : 94 – 108 .[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]). The testing procedure is calibrated by using the subsampling, which is shown to deliver consistent results under weak assumptions on the data generating process and on the structure of the neural network model.     

Variable Selection for Naive Bayes Semisupervised Learning

This article deals with a semisupervised learning based on naive Bayes assumption. A univariate Gaussian mixture density is used for continuous input variables whereas a histogram type density is adopted for discrete input variables. The EM algorithm is used for the computation of maximum likelihood estimators of parameters in the model when we fix the number of mixing components for each continuous input variable. We carry out a model selection for choosing a parsimonious model among various fitted models based on an information criterion. A common density method is proposed for the selection of significant input variables. Simulated and real datasets are used to illustrate the performance of the proposed method.     

A Variable Selection Criterion for Two Sets of Principal Component Scores in Principal Canonical Correlation Analysis

Canonical correlation analysis (CCA) is often used to analyze the correlation between two random vectors. However, sometimes interpretation of CCA results may be hard. In an attempt to address these difficulties, principal canonical correlation analysis (PCCA) was proposed. PCCA is CCA between two sets of principal component (PC) scores. We consider the problem of selecting useful PC scores in CCA. A variable selection criterion for one set of PC scores has been proposed by Ogura (2010), here, we propose a variable selection criterion for two sets of PC scores in PCCA. Furthermore, we demonstrate the effectiveness of this criterion.     

Adaptive Lasso Variable Selection for the Accelerated Failure Models

This article considers the adaptive lasso procedure for the accelerated failure time model with multiple covariates based on weighted least squares method, which uses Kaplan-Meier weights to account for censoring. The adaptive lasso method can complete the variable selection and model estimation simultaneously. Under some mild conditions, the estimator is shown to have sparse and oracle properties. We use Bayesian Information Criterion (BIC) for tuning parameter selection, and a bootstrap variance approach for standard error. Simulation studies and two real data examples are carried out to investigate the performance of the proposed method.     

A New Analysis Strategy for Designs With Complex Aliasing

Abstract

Nonregular designs are popular in planning industrial experiments for their run-size economy. These designs often produce partially aliased effects, where the effects of different factors cannot be completely separated from each other. In this article, we propose applying an adaptive lasso regression as an analytical tool for designs with complex aliasing. Its utility compared to traditional methods is demonstrated by analyzing real-life experimental data and simulation studies.     

Non-iterative Estimation and Variable Selection in the Single-index Quantile Regression Model

A new estimation procedure is proposed for the single-index quantile regression model. Compared to existing work, this approach is non-iterative and hence, computationally efficient. The proposed method not only estimates the index parameter and the link function but also selects variables simultaneously. The performance of the variable selection is enhanced by a fully adaptive penalty function motivated by the sliced inverse regression technique. Finite sample performance is studied through a simulation study that compares the proposed method with existing work under several criteria. A data analysis is given that highlights the usefulness of the proposed methodology.     

Variable Selection for Partially Linear Models with Randomly Censored Data

This article proposes a variable selection procedure for partially linear models with right-censored data via penalized least squares. We apply the SCAD penalty to select significant variables and estimate unknown parameters simultaneously. The sampling properties for the proposed procedure are investigated. The rate of convergence and the asymptotic normality of the proposed estimators are established. Furthermore, the SCAD-penalized estimators of the nonzero coefficients are shown to have the asymptotic oracle property. In addition, an iterative algorithm is proposed to find the solution of the penalized least squares. Simulation studies are conducted to examine the finite sample performance of the proposed method.     

Analytical Characterization of the Information Matrix for Split-Plot Designs

This article provides the analytical characterization of the inverse of the information matrix for second-order SPD. A particular feature of these explicit expressions is that they are functions of the design parameters enabling the development of analytical functions to efficiently compute exact design optimality criteria. The application of these analytical expressions is demonstrated using the generalized variance of the parameter estimates for second-order SPD. An example illustrating the use of these expressions is also presented.     

An Adaptive Method of Variable Selection in Regression

An adaptive variable selection procedure is proposed which uses an adaptive test along with a stepwise procedure to select variables for a multiple regression model. We compared this adaptive stepwise procedure to methods that use Akaike's information criterion, Schwartz's information criterion, and Sawa's information criterion. The simulation studies demonstrated that the adaptive stepwise method is more effective than the traditional variable selection methods if the error distribution is not normally distributed. If the error distribution is known to be normally distributed, the variable selection method based on Sawa's information criteria appears to be superior to the other methods. Unless the error distribution is known to be normally distributed, the adaptive stepwise method is recommended.