The multi-layer information bottleneck problem

Abstract : The muti-layer information bottleneck (IB) problem, where information is propagated (or successively refined) from layer to layer, is considered. Based on information forwarded by the preceding layer, each stage of the network is required to preserve a certain level of relevance with regards to a specific hidden variable, quantified by the mutual information. The hidden variables and the source can be arbitrarily correlated. The optimal trade-off between rates of relevance and compression (or complexity) is obtained through a singleletter characterization, referred to as the rate-relevance region. Conditions of successive refinabilty are given. Binary source with BSC hidden variables and binary source with BSC/BEC mixed hidden variables are both proved to be successively refinable. We further extend our result to Guassian models. A counterexample of successive refinability is also provided.
Complete list of metadatas

https://hal-centralesupelec.archives-ouvertes.fr/hal-01742326
Contributor : Pablo Piantanida <>
Submitted on : Saturday, March 24, 2018 - 2:14:28 PM
Last modification on : Thursday, January 31, 2019 - 10:16:06 AM

Links full text

Identifiers

Citation

Qianqian Yang, Pablo Piantanida, Deniz Gunduz. The multi-layer information bottleneck problem. 2017 IEEE Information Theory Workshop (ITW), Nov 2017, Kaohsiung, France. pp.404-408, ⟨10.1109/ITW.2017.8278006⟩. ⟨hal-01742326⟩

Share

Metrics

Record views

107