Estimating Probability Distributions by Observing Betting Practices

A bookmaker takes bets on a two-horse race, attempting to minimize expected loss over all possible outcomes of the race. Profits are controlled by manipulation of customer' betting behavior; in order to do this, we need some information about the probability distribution which describes how the customers will bet. We examine what information initial customer' betting behavior provides about this probability distribution, and consider how to use this to estimate the probability distribution for remaining customers.     

Modelling Association Football Scores and Inefficiencies in the Football Betting Market

A parametric model is developed and fitted to English league and cup football data from 1992 to 1995. The model is motivated by an aim to exploit potential inefficiencies in the association football betting market, and this is examined using bookmakers' odds from 1995 to 1996. The technique is based on a Poisson regression model but is complicated by the data structure and the dynamic nature of teams' performances. Maximum likelihood estimates are shown to be computationally obtainable, and the model is shown to have a positive return when used as the basis of a betting strategy.     

Efficiency of Betting Markets and Rationality of Players: Evidence from the French 6/49 Lotto

We analyse the existence of preferred numbers on the French Lotto market and prove that this market is not strongly efficient in the sense of Thaler & Ziemba (1988). The preference for low numbers is investigated by means of stochastic dominance tests. The specific features of the French Lotto game allow us to build a simple estimate of the probability distribution of numbers actually played. The results are compared with the (highly time-consuming) maximum likelihood estimator used by Farrell et al. (2000). It is shown that the two methods give very close results. Our conclusions stress the perspectives of this study in various domains.     

Statistics and the medicine of the future

New drugs that are effective and safe for all become harder and harder to find. Individuals react differently to different medications. Chris Harbron points to a future where drugs will be better targeted. With the help of statisticians we will each have our own designer drugs, no longer off-the-shelf but tailored exactly to suit our own individual genome.     

An online classification EM algorithm based on the mixture model

Mixture model-based clustering is widely used in many applications. In certain real-time applications the rapid increase of data size with time makes classical clustering algorithms too slow. An online clustering algorithm based on mixture models is presented in the context of a real-time flaw-diagnosis application for pressurized containers which uses data from acoustic emission signals. The proposed algorithm is a stochastic gradient algorithm derived from the classification version of the EM algorithm (CEM). It provides a model-based generalization of the well-known online k-means algorithm, able to handle non-spherical clusters. Using synthetic and real data sets, the proposed algorithm is compared with the batch CEM algorithm and the online EM algorithm. The three approaches generate comparable solutions in terms of the resulting partition when clusters are relatively well separated, but online algorithms become faster as the size of the available observations increases.     

Recursive online EM estimation of mixture autoregressions

This paper develops a recursive expectation–maximization (REM) algorithm for estimating a mixture autoregression (MAR) with an independent and identically distributed regime transition process. The proposed method, which is useful for long time series as well as for data available in real time, follows a recursive predictor error-type scheme. Based on a slightly modified system to the expectation–maximization (EM) equations for an MAR model, the REM algorithm consists of two steps at each iteration: the expectation step, in which the current unobserved regime transition is estimated from new data using previous recursive estimates, and the minimization step, in which the MAR parameter estimates are recursively updated following a minimization direction. Details of implementation of the REM algorithm are given and its finite-sample performance is shown via simulation experiments. In particular, the EM and REM provide roughly similar estimates, especially for moderate and long time series.     

On future household structure

Summary.  We develop a method for computing probabilistic household forecasts which quantifies uncertainty in the future number of households of various types in a country. A probabilistic household forecast helps policy makers, planners and other forecast users in the fields of housing, energy, social security etc. in taking appropriate decisions, because some household variables are more uncertain than others. Deterministic forecasts traditionally do not quantify uncertainty. We apply the method to data from Norway. We find that predictions of future numbers of married couples, cohabiting couples and one-person households are more certain than those of lone parents and other private households. Our method builds on an existing method for computing probabilistic population forecasts, combining such a forecast with a random breakdown of the population according to household position (single, cohabiting, living with a spouse, living alone etc.). In this application, uncertainty in the total numbers of households of different types derives primarily from random shares, rather than uncertain future population size. A similar method could be applied to obtain probabilistic forecasts for other divisions of the population, such as household size, health or disability status, region of residence and labour market status.     

Multivariate online regression analysis with heterogeneous streaming data

New data collection and storage technologies have given rise to a new field of streaming data analytics, called real-time statistical methodology for online data analyses. Most existing online learning methods are based on homogeneity assumptions, which require the samples in a sequence to be independent and identically distributed. However, inter-data batch correlation and dynamically evolving batch-specific effects are among the key defining features of real-world streaming data such as electronic health records and mobile health data. This article is built under a state-space mixed model framework in which the observed data stream is driven by a latent state process that follows a Markov process. In this setting, online maximum likelihood estimation is made challenging by high-dimensional integrals and complex covariance structures. In this article, we develop a real-time Kalman-filter-based regression analysis method that updates both point estimates and their standard errors for fixed population average effects while adjusting for dynamic hidden effects. Both theoretical justification and numerical experiments demonstrate that our proposed online method has statistical properties similar to those of its offline counterpart and enjoys great computational efficiency. We also apply this method to analyze an electronic health record dataset.     

Distribution-free prediction intervals for the future current record statistics

Prediction of records plays an important role in many applications, such as, meteorology, hydrology, industrial stress testing and athletic events. In this paper, based on the observed current records of an iid sequence sample drawn from an arbitrary unknown distribution, we develop distribution-free prediction intervals as well as prediction upper and lower bounds for current records from another iid sequence. We also present sharp upper bounds for the expected lengths of the so obtained prediction intervals. Numerical computations of the coverage probabilities are presented for choosing the appropriate limits of the prediction intervals.      

Evaluating the ROC performance of markers for future events

Receiver operating characteristic (ROC) curves play a central role in the evaluation of biomarkers and tests for disease diagnosis. Predictors for event time outcomes can also be evaluated with ROC curves, but the time lag between marker measurement and event time must be acknowledged. We discuss different definitions of time-dependent ROC curves in the context of real applications. Several approaches have been proposed for estimation. We contrast retrospective versus prospective methods in regards to assumptions and flexibility, including their capacities to incorporate censored data, competing risks and different sampling schemes. Applications to two datasets are presented.     

Prediction bands for the EDF of a future sample

We develop both nonparametric and parametric methods for obtaining prediction bands for the empirical distribution function (EDF) of a future sample. These methods yield simultaneous prediction intervals for all order statistics of the future sample, and they also correspond to tests for the two-sample problem. The nonparametric prediction bands correspond to the two-sample Kolmogorov-Smirnov test and related nonparametric tests, but the parametric prediction bands correspond to entirely new parametric two-sample tests. The parametric prediction bands tend to outperform the nonparametric bands when the parametric assumptions hold, but they may have true coverage probabilities well below their nominal levels when the parametric assumptions fail. A new computational algorithm is used to obtain critical values in the nonparametric case.