A note on functional linear regression

This work focuses on the linear regression model with functional covariate and scalar response. We compare the performance of two (parametric) linear regression estimators and a nonparametric (kernel) estimator via a Monte Carlo simulation study and the analysis of two real data sets. The first linear estimator expands the predictor and the regression weight function in terms of the trigonometric basis, while the second one uses functional principal components. The choice of the regularization degree in the linear estimators is addressed.     

On the local linear estimate for functional regression: Uniform in bandwidth consistency

We consider the problem of local linear estimation of the regression function when the regressor is functional. The main result of this paper is to prove the strong convergence (with rates), uniformly in bandwidth parameters (UIB), of the considered estimator. The main interest of this result is the possibility to derive the asymptotic properties of our estimate even if the bandwidth parameter is a random variable.     

The historical functional linear model

The authors develop a functional linear model in which the values at time t of a sample of curves yi (t) are explained in a feed‐forward sense by the values of covariate curves xi(s) observed at times s ±.t. They give special attention to the case s ± [t — δ, t], where the lag parameter δ is estimated from the data. They use the finite element method to estimate the bivariate parameter regression function β(s, t), which is defined on the triangular domain s ± t. They apply their model to the problem of predicting the acceleration of the lower lip during speech on the basis of electromyographical recordings from a muscle depressing the lip. They also provide simulation results to guide the calibration of the fitting process.     

Rank estimation for the functional linear model

This article discusses the estimation of the parameter function for a functional linear regression model under heavy-tailed errors' distributions and in the presence of outliers. Standard approaches of reducing the high dimensionality, which is inherent in functional data, are considered. After reducing the functional model to a standard multiple linear regression model, a weighted rank-based procedure is carried out to estimate the regression parameters. A Monte Carlo simulation and a real-world example are used to show the performance of the proposed estimator and a comparison made with the least-squares and least absolute deviation estimators.     

Improved estimation of population mean using known median of auxiliary variable

In this article, we consider the problem of estimation of population mean using the known median of auxiliary variable. We proposed an estimator and its efficiency is studied analytically as well as empirically for different conditions. The proposed estimator is found to be more efficient than traditional estimators such as sample mean and linear regression estimator.     

Classical testing in functional linear models

We extend four tests common in classical regression – Wald, score, likelihood ratio and F tests – to functional linear regression, for testing the null hypothesis, that there is no association between a scalar response and a functional covariate. Using functional principal component analysis, we re-express the functional linear model as a standard linear model, where the effect of the functional covariate can be approximated by a finite linear combination of the functional principal component scores. In this setting, we consider application of the four traditional tests. The proposed testing procedures are investigated theoretically for densely observed functional covariates when the number of principal components diverges. Using the theoretical distribution of the tests under the alternative hypothesis, we develop a procedure for sample size calculation in the context of functional linear regression. The four tests are further compared numerically for both densely and sparsely observed noisy functional data in simulation experiments and using two real data applications.     

Variable selection in partial linear regression with functional covariate

The problem of variable selection is considered in high-dimensional partial linear regression under some model allowing for possibly functional variable. The procedure studied is that of nonconcave-penalized least squares. It is shown the existence of a √n/s_n-consistent estimator for the vector of p_n linear parameters in the model, even when p_n tends to ∞ as the sample size n increases (s_n denotes the number of influential variables). An oracle property is also obtained for the variable selection method, and the nonparametric rate of convergence is stated for the estimator of the nonlinear functional component of the model. Finally, a simulation study illustrates the finite sample size performance of our procedure.     

M-estimation for functional linear regression

This paper studies M-estimation in functional linear regression in which the dependent variable is scalar while the covariate is a function. An estimator for the slope function is obtained based on the functional principal component basis. The global convergence rate of the M-estimator of unknown slope function is established. The convergence rate of the mean-squared prediction error for the proposed estimators is also established. Monte Carlo simulation studies are conducted to examine the finite-sample performance of the proposed procedure. Finally, the proposed method is applied to analyze the Berkeley growth data.     

Estimation and selection procedures in regression: An L1 approach

The authors consider the problem of estimating a regression function go involving several variables by the closest functional element of a prescribed class _G that is closest to it in the L₁ norm. They propose a new estimator ? based on independent observations and give explicit finite sample bounds for the L₁distance between ?g and go. They apply their estimation procedure to the problem of selecting the smoothing parameter in nonparametric regression.     

Electricity consumption prediction with functional linear regression using spline estimators

A functional linear regression model linking observations of a functional response variable with measurements of an explanatory functional variable is considered. This model serves to analyse a real data set describing electricity consumption in Sardinia. The interest lies in predicting either oncoming weekends’ or oncoming weekdays’ consumption, provided actual weekdays’ consumption is known. A B-spline estimator of the functional parameter is used. Selected computational issues are addressed as well.     

Econometric Reviews

This paper considers the problem of estimating a nonlinear statistical model subject to stochastic linear constraints among unknown parameters. These constraints represent prior information which originates from a previous estimation of the same model using an alternative database. One feature of this specification allows for the disign matrix of stochastic linear restrictions to be estimated. The mixed regression technique and the maximum likelihood approach are used to derive the estimator for both the model coefficients and the unknown elements of this design matrix. The proposed estimator whose asymptotic properties are studied, contains as a special case the conventional mixed regression estimator based on a fixed design matrix. A new test of compatibility between prior and sample information is also introduced. Thesuggested estimator is tested empirically with both simulated and actual marketing data.     

Generalized mixed estimator for nonlinear models: a maximum likelihood approach

This paper considers the problem of estimating a nonlinear statistical model subject to stochastic linear constraints among unknown parameters. These constraints represent prior information which originates from a previous estimation of the same model using an alternative database. One feature of this specification allows for the disign matrix of stochastic linear restrictions to be estimated. The mixed regression technique and the maximum likelihood approach are used to derive the estimator for both the model coefficients and the unknown elements of this design matrix. The proposed estimator whose asymptotic properties are studied, contains as a special case the conventional mixed regression estimator based on a fixed design matrix. A new test of compatibility between prior and sample information is also introduced. Thesuggested estimator is tested empirically with both simulated and actual marketing data.     

Optimal estimation in functional linear regression for sparse noise‐contaminated data

In this article, we propose a novel approach to fit a functional linear regression in which both the response and the predictor are functions. We consider the case where the response and the predictor processes are both sparsely sampled at random time points and are contaminated with random errors. In addition, the random times are allowed to be different for the measurements of the predictor and the response functions. The aforementioned situation often occurs in longitudinal data settings. To estimate the covariance and the cross‐covariance functions, we use a regularization method over a reproducing kernel Hilbert space. The estimate of the cross‐covariance function is used to obtain estimates of the regression coefficient function and of the functional singular components. We derive the convergence rates of the proposed cross‐covariance, the regression coefficient, and the singular component function estimators. Furthermore, we show that, under some regularity conditions, the estimator of the coefficient function has a minimax optimal rate. We conduct a simulation study and demonstrate merits of the proposed method by comparing it to some other existing methods in the literature. We illustrate the method by an example of an application to a real‐world air quality dataset. The Canadian Journal of Statistics 47: 524–559; 2019 © 2019 Statistical Society of Canada     

Bootstrapping quantiles in a fixed design regression model with censored data

We consider the problem of estimating the quantiles of a distribution function in a fixed design regression model in which the observations are subject to random right censoring. The quantile estimator is defined via a conditional Kaplan-Meier type estimator for the distribution at a given design point. We establish an a.s. asymptotic representation for this quantile estimator, from which we obtain its asymptotic normality. Because a complicated estimation procedure is necessary for estimating the asymptotic bias and variance, we use a resampling procedure, which provides us, via an asymptotic representation for the bootstrapped estimator, with an alternative for the normal approximation.     

Optimal and Robust Designs for Estimating the Concentration Curve and the AUC

The problem of interest is to estimate the concentration curve and the area under the curve (AUC) by estimating the parameters of a linear regression model with an autocorrelated error process. We introduce a simple linear unbiased estimator of the concentration curve and the AUC. We show that this estimator constructed from a sampling design generated by an appropriate density is asymptotically optimal in the sense that it has exactly the same asymptotic performance as the best linear unbiased estimator. Moreover, we prove that the optimal design is robust with respect to a minimax criterion. When repeated observations are available, this estimator is consistent and has an asymptotic normal distribution. Finally, a simulated annealing algorithm is applied to a pharmacokinetic model with correlated errors.     

Minimax exact constant in sup-norm for nonparametric regression with random design

We consider the nonparametric regression model with random design. We study the estimation of a regression function f in the uniform norm assuming that f belongs to a Hölder class. We determine the minimax exact constant and an asymptotically exact estimator. They depend on the minimum value of the design density.     

Asymptotic normality of locally modelled regression estimator for functional data

We focus on the nonparametric regression of a scalar response on a functional explanatory variable. As an alternative to the well-known Nadaraya-Watson estimator for regression function in this framework, the locally modelled regression estimator performs very well [cf. [Barrientos-Marin, J., Ferraty, F., and Vieu, P. (2010), ‘Locally Modelled Regression and Functional Data’, Journal of Nonparametric Statistics, 22, 617–632]. In this paper, the asymptotic properties of locally modelled regression estimator for functional data are considered. The mean-squared convergence as well as asymptotic normality for the estimator are established. We also adapt the empirical likelihood method to construct the point-wise confidence intervals for the regression function and derive the Wilk's phenomenon for the empirical likelihood inference. Furthermore, a simulation study is presented to illustrate our theoretical results.     

Empirical likelihood confidence intervals for nonparametric functional data analysis

We consider the problem of constructing confidence intervals for nonparametric functional data analysis using empirical likelihood. In this doubly infinite-dimensional context, we demonstrate the Wilk's phenomenon and propose a bias-corrected construction that requires neither undersmoothing nor direct bias estimation. We also extend our results to partially linear regression models involving functional data. Our numerical results demonstrate improved performance of the empirical likelihood methods over normal approximation-based methods.     

Consistency of the recursive nonparametric regression estimation for dependent functional data

We consider the recursive estimation of a regression functional where the explanatory variables take values in some functional space. We prove the almost sure convergence of such estimates for dependent functional data. Also we derive the mean quadratic error of the considered class of estimators. Our results are established with rates and asymptotic appear bounds, under strong mixing condition. Finally, the feasibility of the proposed estimator is illustrated throughout an empirical study.     

Variable selection in linear regression based on ridge estimator

In this article, we consider the problem of variable selection in linear regression when multicollinearity is present in the data. It is well known that in the presence of multicollinearity, performance of least square (LS) estimator of regression parameters is not satisfactory. Consequently, subset selection methods, such as Mallow's Cp, which are based on LS estimates lead to selection of inadequate subsets. To overcome the problem of multicollinearity in subset selection, a new subset selection algorithm based on the ridge estimator is proposed. It is shown that the new algorithm is a better alternative to Mallow's Cp when the data exhibit multicollinearity.