On the foundations: Statistical models and inference

This paper presents an overview of some recent results concerning statistical models and inference. specifically: grounds for statistical models. types of models that simplify by standard probability analysis, the use of categorical information in the reduction of the model with data, and the role of additives in the inference process. The relevant technical material has been developed elsewhere.     

A robust coefficient of determination for regression

To assess the quality of the fit in a multiple linear regression, the coefficient of determination or R² is a very simple tool, yet the most used by practitioners. Indeed, it is reported in most statistical analyzes, and although it is not recommended as a final model selection tool, it provides an indication of the suitability of the chosen explanatory variables in predicting the response. In the classical setting, it is well known that the least-squares fit and coefficient of determination can be arbitrary and/or misleading in the presence of a single outlier. In many applied settings, the assumption of normality of the errors and the absence of outliers are difficult to establish. In these cases, robust procedures for estimation and inference in linear regression are available and provide a suitable alternative.     

Hidden Second‐order Stationary Spatial Point Processes

In the existing statistical literature, the almost default choice for inference on inhomogeneous point processes is the most well‐known model class for inhomogeneous point processes: reweighted second‐order stationary processes. In particular, the K‐function related to this type of inhomogeneity is presented as the inhomogeneous K‐function. In the present paper, we put a number of inhomogeneous model classes (including the class of reweighted second‐order stationary processes) into the common general framework of hidden second‐order stationary processes, allowing for a transfer of statistical inference procedures for second‐order stationary processes based on summary statistics to each of these model classes for inhomogeneous point processes. In particular, a general method to test the hypothesis that a given point pattern can be ascribed to a specific inhomogeneous model class is developed. Using the new theoretical framework, we reanalyse three inhomogeneous point patterns that have earlier been analysed in the statistical literature and show that the conclusions concerning an appropriate model class must be revised for some of the point patterns.     

Higher-order asymptotics under model misspecification

Most of the higher-order asymptotic results in statistical inference available in the literature assume model correctness. The aim of this paper is to develop higher-order results under model misspecification. The density functions to O(n^?3/2) of the robust score test statistic and the robust Wald test statistic are derived under the null hypothesis, for the scalar as well as the multiparameter case. Alternate statistics which are robust to O(n^?3/2) are also proposed.     

Location inference on spheres and cylinders

R.A. Fisher (1957) discussed conditional inference for a spherical normal with mean restricted to a circle. Related statistical models have been discussed by Fraser (1968, 1979). This paper develops statistical inference for a spherical normal with mean restricted to a sphere-cylinder embedded in Rⁿ. This material forms the basis for analyzing non-linear least squares generally, and obtaining second-order data-sensitive conditional procedures.     

Statistical inference on the drift parameter in fractional Brownian motion with a deterministic drift

In statistical inference on the drift parameter a in the fractional Brownian motion W^H_t with the Hurst parameter H ∈ (0, 1) with a constant drift Y^H_t = at + W^H_t, there is a large number of options how to do it. We may, for example, base this inference on the properties of the standard normal distribution applied to the differences between the observed values of the process at discrete times. Although such methods are very simple, it turns out that more appropriate is to use inverse methods. Such methods can be generalized to non constant drift. For the hypotheses testing about the drift parameter a, it is more proper to standardize the observed process, and to use inverse methods based on the first exit time of the observed process of a pre-specified interval until some given time. These procedures are illustrated, and their times of decision are compared against the direct approach. Other generalizations are possible when the random part is a symmetric stochastic integral of a known, deterministic function with respect to fractional Brownian motion.     

Oracally efficient spline-backfitted kernel smoothing of additive partial linear measurement error model

Abstract

We consider statistical inference for additive partial linear models when the linear covariate is measured with error. A bias-corrected spline-backfitted kernel smoothing method is proposed. Under mild assumptions, the proposed component function and parameter estimator are oracally efficient and fast to compute. The nonparametric function estimator’s pointwise distribution is asymptotically equivalent to an function estimator in partial linear model. Finite-sample performance of the proposed estimators is assessed by simulation experiments. The proposed methods are applied to Boston house data set.     

Nonparametric Predictive Inference for Ordinal Data

Nonparametric predictive inference (NPI) is a powerful frequentist statistical framework based only on an exchangeability assumption for future and past observations, made possible by the use of lower and upper probabilities. In this article, NPI is presented for ordinal data, which are categorical data with an ordering of the categories. The method uses a latent variable representation of the observations and categories on the real line. Lower and upper probabilities for events involving the next observation are presented, and briefly compared to NPI for non ordered categorical data. As application, the comparison of multiple groups of ordinal data is presented.     

Restricted Estimation in Partially Linear Errors-in-Variables Models

As a compromise between parametric regression and nonparametric regression, partially linear models are frequently used in statistical modelling. This article considers statistical inference for this semiparametric model when the linear covariate is measured with additive error and some additional linear restrictions on the parametric component are assumed to hold. We propose a restricted corrected profile least-squares estimator for the parametric component, and study the asymptotic normality of the estimator. To test hypothesis on the parametric component, we construct a Wald test statistic and obtain its limiting distribution. Some simulation studies are conducted to illustrate our approaches.     

Liu-type estimator in semiparametric partially linear additive models

Partially linear additive model is useful in statistical modelling as a multivariate nonparametric fitting technique. This paper considers statistical inference for the semiparametric model in the presence of multicollinearity. Based on the profile least-squares (PL) approach and Liu estimation method, we propose a PL Liu estimator for the parametric component. When some additional linear restrictions on the parametric component are available, the corresponding restricted Liu estimator for the parametric component is constructed. The properties of the proposed estimators are derived. Some simulations are conducted to assess the performance of the proposed procedures and the results are satisfactory. Finally, a real data example is analysed.     

Statistical inference for restricted partially linear varying coefficient errors-in-variables models

As a useful extension of partially linear models and varying coefficient models, the partially linear varying coefficient model is useful in statistical modelling. This paper considers statistical inference for the semiparametric model when the covariates in the linear part are measured with additive error and some additional linear restrictions on the parametric component are available. We propose a restricted modified profile least-squares estimator for the parametric component, and prove the asymptotic normality of the proposed estimator. To test hypotheses on the parametric component, we propose a test statistic based on the difference between the corrected residual sums of squares under the null and alterative hypotheses, and show that its limiting distribution is a weighted sum of independent chi-square distributions. We also develop an adjusted test statistic, which has an asymptotically standard chi-squared distribution. Some simulation studies are conducted to illustrate our approaches.     

Marshall–Olkin Extended Lomax Distribution and Its Application to Censored Data

The problem of making statistical inference about θ =P(X > Y) has been under great investigation in the literature using simple random sampling (SRS) data. This problem arises naturally in the area of reliability for a system with strength X and stress Y. In this study, we will consider making statistical inference about θ using ranked set sampling (RSS) data. Several estimators are proposed to estimate θ using RSS. The properties of these estimators are investigated and compared with known estimators based on simple random sample (SRS) data. The proposed estimators based on RSS dominate those based on SRS. A motivated example using real data set is given to illustrate the computation of the newly suggested estimators.     

A generalized framework for modelling ordinal data

In several applied disciplines, as Economics, Marketing, Business, Sociology, Psychology, Political science, Environmental research and Medicine, it is common to collect data in the form of ordered categorical observations. In this paper, we introduce a class of models based on mixtures of discrete random variables in order to specify a general framework for the statistical analysis of this kind of data. The structure of these models allows the interpretation of the final response as related to feeling, uncertainty and a possible shelter option and the expression of the relationship among these components and subjects’ covariates. Such a model may be effectively estimated by maximum likelihood methods leading to asymptotically efficient inference. We present a simulation experiment and discuss a real case study to check the consistency and the usefulness of the approach. Some final considerations conclude the paper.     

The Scaled Uniform Model Revisited

Abstract

Sufficiency, conditionality, and invariance are basic principles of statistical inference. Current mathematical statistics courses do not devote much teaching time to these classical principles, and even ignore the latter two, in order to teach modern methods. However, being the philosophical cornerstones of statistical inference, a minimal understanding of these principles should be part of any curriculum in statistics. The scaled uniform model is used here to demonstrate the importance and usefulness of the conditionality principle, which is probably the most basic and less familiar among the three.     

Theoretic and empirical data-inclusive process characterization

Summary.  In process characterization the quality of information that is obtained depends directly on the quality of process model. The current quality revolution is now providing a strong stimulus for rethinking and re-evaluating many statistical ideas. Among these are the role of theoretic knowledge and data in statistical inference and some issues in theoretic–empirical modelling. With this concern the paper takes a broad, pragmatic view of statistical inference to include all aspects of model formulation. The estimation of model parameters traditionally assumes that a model has a prespecified known form and takes no account of possible uncertainty regarding model structure. But in practice model structural uncertainty is a fact of life and is likely to be more serious than other sources of uncertainty which have received far more attention. This is true whether the model is specified on subject-matter grounds or when a model is formulated, fitted and checked on the same data set in an iterative interactive way. For that reason novel modelling techniques have been fashioned for reducing model uncertainty. Using available knowledge for theoretic model elaboration the techniques that have been created approximate the exact unknown process model concurrently by accessible theoretic and polynomial empirical functions. The paper examines the effects of uncertainty for hybrid theoretic–empirical models and, for reducing uncertainty, additive and multiplicative methods of model formulation are fashioned. Such modelling techniques have been successfully applied to perfect a steady flow model for an air gauge sensor. Validation of the models elaborated has revealed that the multiplicative modelling approach allows us to attain a satisfactory model with small discrepancy from empirical evidence.     

Discretisation for inference on normal mixture models

The problem of inference in Bayesian Normal mixture models is known to be difficult. In particular, direct Bayesian inference (via quadrature) suffers from a combinatorial explosion in having to consider every possible partition of n observations into k mixture components, resulting in a computation time which is O(k ⁿ). This paper explores the use of discretised parameters and shows that for equal-variance mixture models, direct computation time can be reduced to O(D ^k n ^k), where relevant continuous parameters are each divided into D regions. As a consequence, direct inference is now possible on genuine data sets for small k, where the quality of approximation is determined by the level of discretisation. For large problems, where the computational complexity is still too great in O(D ^k n ^k) time, discretisation can provide a convergence diagnostic for a Markov chain Monte Carlo analysis.     

Statistical Inference Enables Bad Science; Statistical Thinking Enables Good Science

ABSTRACT

Scientific research of all kinds should be guided by statistical thinking: in the design and conduct of the study, in the disciplined exploration and enlightened display of the data, and to avoid statistical pitfalls in the interpretation of the results. However, formal, probability-based statistical inference should play no role in most scientific research, which is inherently exploratory, requiring flexible methods of analysis that inherently risk overfitting. The nature of exploratory work is that data are used to help guide model choice, and under these circumstances, uncertainty cannot be precisely quantified, because of the inevitable model selection bias that results. To be valid, statistical inference should be restricted to situations where the study design and analysis plan are specified prior to data collection. Exploratory data analysis provides the flexibility needed for most other situations, including statistical methods that are regularized, robust, or nonparametric. Of course, no individual statistical analysis should be considered sufficient to establish scientific validity: research requires many sets of data along many lines of evidence, with a watchfulness for systematic error. Replicating and predicting findings in new data and new settings is a stronger way of validating claims than blessing results from an isolated study with statistical inferences.     

The Limited Role of Formal Statistical Inference in Scientific Inference

ABSTRACT

Such is the grip of formal methods of statistical inference—that is, frequentist methods for generalizing from sample to population in enumerative studies—in the drawing of scientific inferences that the two are routinely deemed equivalent in the social, management, and biomedical sciences. This, despite the fact that legitimate employment of said methods is difficult to implement on practical grounds alone. But supposing the adoption of these procedures were simple does not get us far; crucially, methods of formal statistical inference are ill-suited to the analysis of much scientific data. Even findings from the claimed gold standard for examination by the latter, randomized controlled trials, can be problematic.

Scientific inference is a far broader concept than statistical inference. Its authority derives from the accumulation, over an extensive period of time, of both theoretical and empirical knowledge that has won the (provisional) acceptance of the scholarly community. A major focus of scientific inference can be viewed as the pursuit of significant sameness, meaning replicable and empirically generalizable results among phenomena. Regrettably, the obsession with users of statistical inference to report significant differences in data sets actively thwarts cumulative knowledge development.

The manifold problems surrounding the implementation and usefulness of formal methods of statistical inference in advancing science do not speak well of much teaching in methods/statistics classes. Serious reflection on statistics' role in producing viable knowledge is needed. Commendably, the American Statistical Association is committed to addressing this challenge, as further witnessed in this special online, open access issue of The American Statistician.     

Bayesian inference over ICA models: application to multibiometric score fusion with quality estimates

Bayesian networks are not well-formulated for continuous variables. The majority of recent works dealing with Bayesian inference are restricted only to special types of continuous variables such as the conditional linear Gaussian model for Gaussian variables. In this context, an exact Bayesian inference algorithm for clusters of continuous variables which may be approximated by independent component analysis models is proposed. The complexity in memory space is linear and the overfitting problem is attenuated, while the inference time is still exponential. Experiments for multibiometric score fusion with quality estimates are conducted, and it is observed that the performances are satisfactory compared to some known fusion techniques.     

Importance sampling for signal extraction model in time series analysis

In time series analysis, signal extraction model (SEM) is used to estimate unobserved signal component from observed time series data. Since parameters of the components in SEM are often unknown in practice, a commonly used method is to estimate unobserved signal component using the maximum likelihood estimates (MLEs) of parameters of the components. This paper explores an alternative way to estimate unobserved signal component when parameters of the components are unknown. The suggested method makes use of importance sampling (IS) with Bayesian inference. The basic idea is to treat parameters of the components in SEM as a random vector and compute a posterior probability density function of the parameters using Bayesian inference. Then IS method is applied to integrate out the parameters and thus estimates of unobserved signal component, unconditional to the parameters, can be obtained. This method is illustrated with a real time series data. Then a Monte Carlo study with four different types of time series models is carried out to compare a performance of this method with that of a commonly used method. The study shows that IS method with Bayesian inference is computationally feasible and robust, and more efficient in terms of mean square errors (MSEs) than a commonly used method.