Improving the estimators of the parameters of a probit regression model: A ridge regression approach

This paper considered the estimation of the regression parameters of a general probit regression model. Accordingly, we proposed five ridge regression (RR) estimators for the probit regression models for estimating the parameters (β)$(\beta )$ when the weighted design matrix is ill-conditioned and it is suspected that the parameter β$\beta$ may belong to a linear subspace defined by Hβ=h$H\beta =h$. Asymptotic properties of the estimators are studied with respect to quadratic biases, MSE matrices and quadratic risks. The regions of optimality of the proposed estimators are determined based on the quadratic risks. Some relative efficiency tables and risk graphs are provided to illustrate the numerical comparison of the estimators. We conclude that when q≥3$q\ge 3$, one would uses PRRRE; otherwise one uses PTRRE with some optimum size α$\alpha$. We also discuss the performance of the proposed estimators compare to the alternative ridge regression method due to Liu (1993).     

Testing for random effects and serial correlation in spatial autoregressive models

This paper constructs and evaluates tests for random effects and serial correlation in spatial autoregressive panel data models. In these models, ignoring the presence of random effects not only produces misleading inference but inconsistent estimation of the regression coefficients. Two different estimation methods are considered: maximum likelihood and instrumental variables. For each estimator, optimal tests are constructed: Lagrange multiplier in the first case; Neyman's C(α)$C(\alpha )$ in the second. In addition, locally size-robust tests, for individual hypotheses under local misspecification of the unconsidered parameter, are constructed. Extensive Monte Carlo evidence is presented.     

Estimation of smooth regression functions in monotone response models

We consider the estimation of smooth regression functions in a class of conditionally parametric co-variate-response models. Independent and identically distributed observations are available from the distribution of (Z,X)$(Z,X)$, where Z is a real-valued co-variate with some unknown distribution, and the response X conditional on Z   is distributed according to the density p(·,ψ(Z))$p(·,\psi (Z))$, where p(·,θ)$p(·,\theta )$ is a one-parameter exponential family. The function ψ$\psi$ is a smooth monotone function. Under this formulation, the regression function E(X|Z)$E(X|Z)$ is monotone in the co-variate Z   (and can be expressed as a one–one function of ψ$\psi$); hence the term “monotone response model”. Using a penalized least squares approach that incorporates both monotonicity and smoothness, we develop a scheme for producing smooth monotone estimates of the regression function and also the function ψ$\psi$ across this entire class of models. Point-wise asymptotic normality of this estimator is established, with the rate of convergence depending on the smoothing parameter. This enables construction of Wald-type (point-wise) as well as pivotal confidence sets for ψ$\psi$ and also the regression function. The methodology is extended to the general heteroscedastic model, and its asymptotic properties are discussed.     

Asymptotic results in canonical discriminant analysis when the dimension is large compared to the sample size

This paper deals with the distributions of test statistics for the number of useful discriminant functions and the characteristic roots in canonical discriminant analysis. These asymptotic distributions have been extensively studied when the number p   of variables is fixed, the number q+1$q+1$ of groups is fixed, and the sample size N tends to infinity. However, these approximations become increasingly inaccurate as the value of p increases for a fixed value of N. On the other hand, we encounter to analyze high-dimensional data such that p is large compared to n. The purpose of the present paper is to derive asymptotic distributions of these statistics in a high-dimensional framework such that q   is fixed, p→∞$p\to \infty$, m=n-p+q→∞$m=n-p+q\to \infty$, and p/n→c∈(0,1)$p/n\to c\in (0,1)$, where n=N-q-1$n=N-q-1$. Numerical simulation revealed that our new asymptotic approximations are more accurate than the classical asymptotic approximations in a considerably wide range of (n,p,q)$(n,p,q)$.     

Conditional symmetry models for three-way contingency tables

We generalize the classical conditional or triangular symmetry model for I×I$I\times I$ contingency tables to three-way I×I×I$I\times I\times I$ tables with commensurable ordinal classification variables. The construction of the new family of models is such that the desirable property that connects conditional symmetry to complete symmetry and marginal homogeneity models in two-way tables is retained in three-way tables. Furthermore, connections between our proposed models obey a coherent structure. We provide maximum likelihood estimation for the new models which is illustrated with a real data example.     

Statistical analysis for rounded data

When random variables do not take discrete values, observed data are often the rounded values of continuous random variables. Errors caused by rounding of data are often neglected by classical statistical theories. While some pioneers have identified and made suggestions to rectify the problem, few suitable approaches were proposed. In this paper, we propose an approximate MLE (AMLE) procedure to estimate the parameters and discuss the consistency and asymptotic normality of the estimates. For our illustration, we shall consider the estimates of the parameters in AR(p)$\mathit{AR}(p)$ and MA(q)$\mathit{MA}(q)$ models for rounded data.     

Efficiency in estimation of memory

We study the efficiency of semiparametric estimates of memory parameter. We propose a class of shift invariant tapers of order (p,q). For a fixed p, the variance inflation factor of the new tapers approaches 1 as q   goes to infinity. We show that for d∈(−1/2,p+1/2)$d\in (-1/2,p+1/2)$, the proposed tapered Gaussian semiparametric estimator has the same limiting distribution as the nontapered version for d∈(−1/2,1/2)$d\in (-1/2,1/2)$. The new estimator is mean and polynomial trend invariant, and is computationally advantageous in comparison to the recently proposed exact local Whittle estimator. The simulation study shows that our estimator has comparable or better mean squared error in finite samples for a variety of models.     

Markov chain Monte Carlo tests for designed experiments

We consider conditional exact tests of factor effects in designed experiments for discrete response variables. Similarly to the analysis of contingency tables, a Markov chain Monte Carlo method can be used for performing exact tests, when large-sample approximations are poor and the enumeration of the conditional sample space is infeasible. For designed experiments with a single observation for each run, we formulate log-linear or logistic models and consider a connected Markov chain over an appropriate sample space. In particular, we investigate fractional factorial designs with 2^p-q$2^{p-q}$ runs, noting correspondences to the models for 2^p-q$2^{p-q}$ contingency tables.     

Run orders for efficient two level experimental plans with minimum factor level changes robust to time trends

In this paper we construct run orders of orthogonal arrays with 12≤n≤28$12\le n\le 28$ runs and 4≤q≤6$4\le q\le 6$ factors that minimize the number of level changes of each factor. The corresponding orthogonal arrays can estimate a model with all main effects and their two factor interactions with the highest efficiency and also provide estimates of all main effects that are independent of linear and quadratic time (or position) trends. Some alternative efficient run orders are also presented when the estimation of two factor interactions is of experimental interest.     

Exact D-optimal designs for a linear log contrast model with mixture experiment for three and four ingredients

This study investigates the exact D-optimal designs of the linear log contrast model using the mixture experiment suggested by Aitchison and Bacon-Shone (1984) and the design space restricted by Lim (1987) and Chan (1988). Results show that for three ingredients, there are six extreme points that can be divided into two non-intersect sets S₁ and S₂. An exact N-point D  -optimal design for N=3p+q,p≥1,1≤q≤2$N=3p+q,p\ge 1,1\le q\le 2$ arranges equal weight n/N,0≤n≤p$n/N,0\le n\le p$ at the points of S₁ (S₂) and puts the remaining weight (N−3n)/N$(N-3n)/N$ on the points of S₂ (S₁) as evenly as possible. For four ingredients and N=6p+q,p≥1,1≤q≤5$N=6p+q,p\ge 1,1\le q\le 5$, an exact N-point design that distributes the weights as evenly as possible among the six supports of the approximate D-optimal design is exact D-optimal.     

Efficiency of weighted averages

When combining estimates of a common parameter (of dimension d?1$d?1$) from independent data sets—as in stratified analyses and meta analyses—a weighted average, with weights ‘proportional’ to inverse variance matrices, is shown to have a minimal variance matrix (a standard fact when d=1$d=1$)—minimal in the sense that all convex combinations of the coordinates of the combined estimate have minimal variances. Minimum variance for the estimation of a single coordinate of the parameter can therefore be achieved by joint estimation of all coordinates using matrix weights. Moreover, if each estimate is asymptotically efficient within its own data set, then this optimally weighted average, with consistently estimated weights, is shown to be asymptotically efficient in the combined data set and avoids the need to merge the data sets and estimate the parameter in question afresh. This is so whatever additional non-common nuisance parameters may be in the models for the various data sets. A special case of this appeared in Fisher [1925. Theory of statistical estimation. Proc. Cambridge Philos. Soc. 22, 700–725.]: Optimal weights are ‘proportional’ to information matrices, and he argued that sample information should be used as weights rather than expected information, to maintain second-order efficiency of maximum likelihood. A number of special cases have appeared in the literature; we review several of them and give additional special cases, including stratified regression analysis—proportional-hazards, logistic or linear—, combination of independent ROC curves, and meta analysis. A test for homogeneity of the parameter across the data sets is also given.     

Confidence intervals in regression utilizing prior information

We consider a linear regression model with regression parameter β=(_β1,…,_βp)$\beta =({\beta }_1,\dots ,{\beta }_p)$ and independent and identically N(0,σ²)$N(0,{\sigma }^2)$ distributed errors. Suppose that the parameter of interest is θ=a^Tβ$\theta =a^T\beta$ where a$a$ is a specified vector. Define the parameter τ=c^Tβ-t$\tau =c^T\beta -t$ where the vector c$c$ and the number t$t$ are specified and a$a$ and c$c$ are linearly independent. Also suppose that we have uncertain prior information that τ=0$\tau =0$. We present a new frequentist 1-α$1-\alpha$ confidence interval for θ$\theta$ that utilizes this prior information. We require this confidence interval to (a) have endpoints that are continuous functions of the data and (b) coincide with the standard 1-α$1-\alpha$ confidence interval when the data strongly contradict this prior information. This interval is optimal in the sense that it has minimum weighted average expected length where the largest weight is given to this expected length when τ=0$\tau =0$. This minimization leads to an interval that has the following desirable properties. This interval has expected length that (a) is relatively small when the prior information about τ$\tau$ is correct and (b) has a maximum value that is not too large. The following problem will be used to illustrate the application of this new confidence interval. Consider a 2×2$2\times 2$ factorial experiment with 20 replicates. Suppose that the parameter of interest θ$\theta$ is a specified simple   effect and that we have uncertain prior information that the two-factor interaction is zero. Our aim is to find a frequentist 0.95 confidence interval for θ$\theta$ that utilizes this prior information.     

Grouping strategies and thresholding for high dimensional linear models

The estimation problem in a high regression model with structured sparsity is investigated. An algorithm using a two-step block thresholding procedure called GR-LOL is provided. Convergence rates are produced: they depend on simple coherence-type indices of the Gram matrix – easily checkable on the data – as well as sparsity assumptions of the model parameters measured by a combination of l₁ within-blocks with _lq$l_q$, q<1$q<1$ between-blocks norms. The simplicity of the coherence indicator suggests ways to optimize the rates of convergence when the group structure is not naturally given by the problem or is unknown. In such a case, an auto-driven procedure is provided to determine the regressor groups (number and contents). An intensive practical study compares our grouping methods with the standard LOL algorithm. We prove that the grouping rarely deteriorates the results but can improve them very significantly. GR-LOL is also compared with group-Lasso procedures and exhibits a very encouraging behavior. The results are quite impressive, especially when GR-LOL algorithm is combined with a grouping pre-processing.