Skip to Main content Skip to Navigation
Book sections

This is IT: A Primer on Shannon’s Entropy and Information

Abstract : What is Shannon’s information theory (IT)? Despite its continued impact on our digital society, Claude Shannon’s life and work is still unknown to numerous people. In this tutorial, we review many aspects of the concept of entropy and information from a historical and mathematical point of view. The text is structured into small, mostly independent sections, each covering a particular topic. For simplicity we restrict our attention to one-dimensional variables and use logarithm and exponential notations log and exp without specifying the base. We culminate with a simple exposition of a recent proof (2017) of the entropy power inequality (EPI), one of the most fascinating inequalities in the theory.
Complete list of metadata
Contributor : Olivier Rioul Connect in order to contact the contributor
Submitted on : Monday, August 30, 2021 - 11:48:54 AM
Last modification on : Tuesday, October 19, 2021 - 11:14:21 AM


rioul cropped.pdf
Files produced by the author(s)


  • HAL Id : hal-03326385, version 1


Olivier Rioul. This is IT: A Primer on Shannon’s Entropy and Information. Bertrand Duplantier and Vincent Rivasseau. Information Theory Poincaré Seminar 2018, 78, Birkhäuser, Springer Nature, 2021, Progress in Mathematical Physics, 978-3-030-81479-3. ⟨hal-03326385⟩



Record views


Files downloads