A learning algorithm with compression-based regularization

Abstract : This paper investigates, from information theoretic principles, a learning problem based on the principle that any regularity in a given dataset can be exploited to extract compact features from data, in order to build meaningful representations of a relevant content. We begin by introducing the fundamental tradeoff between the average risk and the model complexity. Interestingly, our formulation allows an information theoretic formulation of the multi-task learning (MTL) problem. Then, we present an iterative algorithm for computing the optimal tradeoffs. Remarkably, empirical results illustrate that there exists an optimal information rate minimizing the excess risk which depends on the nature and the amount of available training data. An application to hierarchical text categorization is also investigated, extending previous works.
Type de document :
Communication dans un congrès
IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2018), Apr 2018, Calgary, Alberta, Canada. 〈10.1109/icassp.2018.8461441 〉
Liste complète des métadonnées

https://hal-centralesupelec.archives-ouvertes.fr/hal-01742449
Contributeur : Pablo Piantanida <>
Soumis le : dimanche 25 mars 2018 - 00:04:23
Dernière modification le : mardi 20 novembre 2018 - 16:16:04

Identifiants

Citation

Matias Vera, Pablo Piantanida, Leonardo Rey Vega. A learning algorithm with compression-based regularization. IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2018), Apr 2018, Calgary, Alberta, Canada. 〈10.1109/icassp.2018.8461441 〉. 〈hal-01742449〉

Partager

Métriques

Consultations de la notice

102