TY - JOUR A1 - Katz-Samuels, Julian A1 - Blanchard, Gilles A1 - Scott, Clayton T1 - Decontamination of Mutual Contamination Models JF - Journal of machine learning research N2 - Many machine learning problems can be characterized by mutual contamination models. In these problems, one observes several random samples from different convex combinations of a set of unknown base distributions and the goal is to infer these base distributions. This paper considers the general setting where the base distributions are defined on arbitrary probability spaces. We examine three popular machine learning problems that arise in this general setting: multiclass classification with label noise, demixing of mixed membership models, and classification with partial labels. In each case, we give sufficient conditions for identifiability and present algorithms for the infinite and finite sample settings, with associated performance guarantees. KW - multiclass classification with label noise KW - classification with partial labels KW - mixed membership models KW - topic modeling KW - mutual contamination models Y1 - 2019 UR - http://arxiv.org/abs/1710.01167 SN - 1532-4435 VL - 20 PB - Microtome Publishing CY - Cambridge, Mass. ER - TY - JOUR A1 - Bachoc, Francois A1 - Blanchard, Gilles A1 - Neuvial, Pierre T1 - On the post selection inference constant under restricted isometry properties JF - Electronic journal of statistics N2 - Uniformly valid confidence intervals post model selection in regression can be constructed based on Post-Selection Inference (PoSI) constants. PoSI constants are minimal for orthogonal design matrices, and can be upper bounded in function of the sparsity of the set of models under consideration, for generic design matrices. In order to improve on these generic sparse upper bounds, we consider design matrices satisfying a Restricted Isometry Property (RIP) condition. We provide a new upper bound on the PoSI constant in this setting. This upper bound is an explicit function of the RIP constant of the design matrix, thereby giving an interpolation between the orthogonal setting and the generic sparse setting. We show that this upper bound is asymptotically optimal in many settings by constructing a matching lower bound. KW - Inference post model-selection KW - confidence intervals KW - PoSI constants KW - linear regression KW - high-dimensional inference KW - sparsity KW - restricted isometry property Y1 - 2018 U6 - https://doi.org/10.1214/18-EJS1490 SN - 1935-7524 VL - 12 IS - 2 SP - 3736 EP - 3757 PB - Institute of Mathematical Statistics CY - Cleveland ER - TY - JOUR A1 - Blanchard, Gilles A1 - Mücke, Nicole T1 - Kernel regression, minimax rates and effective dimensionality BT - beyond the regular case JF - Analysis and applications N2 - We investigate if kernel regularization methods can achieve minimax convergence rates over a source condition regularity assumption for the target function. These questions have been considered in past literature, but only under specific assumptions about the decay, typically polynomial, of the spectrum of the the kernel mapping covariance operator. In the perspective of distribution-free results, we investigate this issue under much weaker assumption on the eigenvalue decay, allowing for more complex behavior that can reflect different structure of the data at different scales. KW - Kernel regression KW - minimax optimality KW - eigenvalue decay Y1 - 2020 U6 - https://doi.org/10.1142/S0219530519500258 SN - 0219-5305 SN - 1793-6861 VL - 18 IS - 4 SP - 683 EP - 696 PB - World Scientific CY - New Jersey ER - TY - BOOK A1 - Blanchard, Gilles T1 - Komplexitätsanalyse in Statistik und Lerntheorie : Antrittsvorlesung 2011-05-04 N2 - Gilles Blanchards Vortrag gewährt Einblicke in seine Arbeiten zur Entwicklung und Analyse statistischer Eigenschaften von Lernalgorithmen. In vielen modernen Anwendungen, beispielsweise bei der Schrifterkennung oder dem Spam- Filtering, kann ein Computerprogramm auf der Basis vorgegebener Beispiele automatisch lernen, relevante Vorhersagen für weitere Fälle zu treffen. Mit der mathematischen Analyse der Eigenschaften solcher Methoden beschäftigt sich die Lerntheorie, die mit der Statistik eng zusammenhängt. Dabei spielt der Begriff der Komplexität der erlernten Vorhersageregel eine wichtige Rolle. Ist die Regel zu einfach, wird sie wichtige Einzelheiten ignorieren. Ist sie zu komplex, wird sie die vorgegebenen Beispiele "auswendig" lernen und keine Verallgemeinerungskraft haben. Blanchard wird erläutern, wie Mathematische Werkzeuge dabei helfen, den richtigen Kompromiss zwischen diesen beiden Extremen zu finden. Y1 - 2011 UR - http://info.ub.uni-potsdam.de/multimedia/show_multimediafile.php?mediafile_id=551 PB - Univ.-Bibl. CY - Potsdam ER - TY - JOUR A1 - Blanchard, Gilles A1 - Dickhaus, Thorsten A1 - Roquain, Etienne A1 - Villers, Fanny T1 - On least favorable configurations for step-up-down tests JF - Statistica Sinica KW - False discovery rate KW - least favorable configuration KW - multiple testing; Y1 - 2014 U6 - https://doi.org/10.5705/ss.2011.205 SN - 1017-0405 SN - 1996-8507 VL - 24 IS - 1 SP - 1 EP - U31 PB - Statistica Sinica, Institute of Statistical Science, Academia Sinica CY - Taipei ER - TY - JOUR A1 - Blanchard, Gilles A1 - Mathe, Peter T1 - Discrepancy principle for statistical inverse problems with application to conjugate gradient iteration JF - Inverse problems : an international journal of inverse problems, inverse methods and computerised inversion of data N2 - The authors discuss the use of the discrepancy principle for statistical inverse problems, when the underlying operator is of trace class. Under this assumption the discrepancy principle is well defined, however a plain use of it may occasionally fail and it will yield sub-optimal rates. Therefore, a modification of the discrepancy is introduced, which corrects both of the above deficiencies. For a variety of linear regularization schemes as well as for conjugate gradient iteration it is shown to yield order optimal a priori error bounds under general smoothness assumptions. A posteriori error control is also possible, however at a sub-optimal rate, in general. This study uses and complements previous results for bounded deterministic noise. Y1 - 2012 U6 - https://doi.org/10.1088/0266-5611/28/11/115011 SN - 0266-5611 VL - 28 IS - 11 PB - IOP Publ. Ltd. CY - Bristol ER - TY - JOUR A1 - Blanchard, Gilles A1 - Delattre, Sylvain A1 - Roquain, Etienne T1 - Testing over a continuum of null hypotheses with False Discovery Rate control JF - Bernoulli : official journal of the Bernoulli Society for Mathematical Statistics and Probability N2 - We consider statistical hypothesis testing simultaneously over a fairly general, possibly uncountably infinite, set of null hypotheses, under the assumption that a suitable single test (and corresponding p-value) is known for each individual hypothesis. We extend to this setting the notion of false discovery rate (FDR) as a measure of type I error. Our main result studies specific procedures based on the observation of the p-value process. Control of the FDR at a nominal level is ensured either under arbitrary dependence of p-values, or under the assumption that the finite dimensional distributions of the p-value process have positive correlations of a specific type (weak PRDS). Both cases generalize existing results established in the finite setting. Its interest is demonstrated in several non-parametric examples: testing the mean/signal in a Gaussian white noise model, testing the intensity of a Poisson process and testing the c.d.f. of i.i.d. random variables. KW - continuous testing KW - false discovery rate KW - multiple testing KW - positive correlation KW - step-up KW - stochastic process Y1 - 2014 U6 - https://doi.org/10.3150/12-BEJ488 SN - 1350-7265 SN - 1573-9759 VL - 20 IS - 1 SP - 304 EP - 333 PB - International Statistical Institute CY - Voorburg ER - TY - JOUR A1 - Kloft, Marius A1 - Blanchard, Gilles T1 - On the Convergence Rate of l(p)-Norm Multiple Kernel Learning JF - JOURNAL OF MACHINE LEARNING RESEARCH N2 - We derive an upper bound on the local Rademacher complexity of l(p)-norm multiple kernel learning, which yields a tighter excess risk bound than global approaches. Previous local approaches analyzed the case p - 1 only while our analysis covers all cases 1 <= p <= infinity, assuming the different feature mappings corresponding to the different kernels to be uncorrelated. We also show a lower bound that shows that the bound is tight, and derive consequences regarding excess loss, namely fast convergence rates of the order O( n(-)1+alpha/alpha where alpha is the minimum eigenvalue decay rate of the individual kernels. KW - multiple kernel learning KW - learning kernels KW - generalization bounds KW - local Rademacher complexity Y1 - 2012 SN - 1532-4435 VL - 13 SP - 2465 EP - 2502 PB - MICROTOME PUBL CY - BROOKLINE ER - TY - JOUR A1 - Beinrucker, Andre A1 - Dogan, Urun A1 - Blanchard, Gilles T1 - Extensions of stability selection using subsamples of observations and covariates JF - Statistics and Computing N2 - We introduce extensions of stability selection, a method to stabilise variable selection methods introduced by Meinshausen and Buhlmann (J R Stat Soc 72:417-473, 2010). We propose to apply a base selection method repeatedly to random subsamples of observations and subsets of covariates under scrutiny, and to select covariates based on their selection frequency. We analyse the effects and benefits of these extensions. Our analysis generalizes the theoretical results of Meinshausen and Buhlmann (J R Stat Soc 72:417-473, 2010) from the case of half-samples to subsamples of arbitrary size. We study, in a theoretical manner, the effect of taking random covariate subsets using a simplified score model. Finally we validate these extensions on numerical experiments on both synthetic and real datasets, and compare the obtained results in detail to the original stability selection method. KW - Variable selection KW - Stability selection KW - Subsampling Y1 - 2016 U6 - https://doi.org/10.1007/s11222-015-9589-y SN - 0960-3174 SN - 1573-1375 VL - 26 SP - 1059 EP - 1077 PB - Springer CY - Dordrecht ER - TY - JOUR A1 - Mieth, Bettina A1 - Kloft, Marius A1 - Rodriguez, Juan Antonio A1 - Sonnenburg, Soren A1 - Vobruba, Robin A1 - Morcillo-Suarez, Carlos A1 - Farre, Xavier A1 - Marigorta, Urko M. A1 - Fehr, Ernst A1 - Dickhaus, Thorsten A1 - Blanchard, Gilles A1 - Schunk, Daniel A1 - Navarro, Arcadi A1 - Müller, Klaus-Robert T1 - Combining Multiple Hypothesis Testing with Machine Learning Increases the Statistical Power of Genome-wide Association Studies JF - Scientific reports N2 - The standard approach to the analysis of genome-wide association studies (GWAS) is based on testing each position in the genome individually for statistical significance of its association with the phenotype under investigation. To improve the analysis of GWAS, we propose a combination of machine learning and statistical testing that takes correlation structures within the set of SNPs under investigation in a mathematically well-controlled manner into account. The novel two-step algorithm, COMBI, first trains a support vector machine to determine a subset of candidate SNPs and then performs hypothesis tests for these SNPs together with an adequate threshold correction. Applying COMBI to data from a WTCCC study (2007) and measuring performance as replication by independent GWAS published within the 2008-2015 period, we show that our method outperforms ordinary raw p-value thresholding as well as other state-of-the-art methods. COMBI presents higher power and precision than the examined alternatives while yielding fewer false (i.e. non-replicated) and more true (i.e. replicated) discoveries when its results are validated on later GWAS studies. More than 80% of the discoveries made by COMBI upon WTCCC data have been validated by independent studies. Implementations of the COMBI method are available as a part of the GWASpi toolbox 2.0. Y1 - 2016 U6 - https://doi.org/10.1038/srep36671 SN - 2045-2322 VL - 6 PB - Nature Publ. Group CY - London ER -