This book provides readers with the fundamentals of information theoretic techniques for statistical data science analyses and for characterizing the behavior and performance of a learning agent outside of the standard results on communications and compression fundamental limits. Readers will benefit from the presentation of information theoretic quantities, definitions, and results that provide or could provide insights into data science and learning.

Describes the fundamentals of information theoretic techniques for statistical data science analyses Provides succinct introductions to key topics, with references as needed for further technical depth Enables readers from varying backgrounds to understand the behavior and performance of a learning agent

Autorentext

Jerry D. Gibson is Professor of Electrical and Computer Engineering at the University of California, Santa Barbara. He is co-author of the books Digital Compression for Multimedia (Morgan-Kaufmann, 1998) and Introduction to Nonparametric Detection with Applications (Academic Press, 1975 and IEEE Press, 1995) and author of the textbook, Principles of Digital and Analog Communications (Prentice-Hall, second ed., 1993). He is Editor-in-Chief of The Mobile Communications Handbook (CRC Press, 3rd ed., 2012), Editor-in-Chief of The Communications Handbook (CRC Press, 2nd ed., 2002), and Editor of the book, Multimedia Communications: Directions and Innovations (Academic Press, 2000). His most recent books are Rate Distortion Bounds for Voice and Video (Coauthor with Jing Hu, NOW Publishers, 2014), and Information Theory and Rate Distortion Theory for Communications and Compression (Morgan-Claypool, 2014). Dr. Gibson was Associate Editor for Speech Processing for the IEEE Transactions on Communications from 1981 to 1985 and Associate Editor for Communications for the IEEE Transactions on Information Theory from 1988-1991. He was an IEEE Communications Society Distinguished Lecturer for 2007-2008. In 1990, Dr. Gibson received The Fredrick Emmons Terman Award from the American Society for Engineering Education, and in 1992, he was elected Fellow of the IEEE. He was the recipient of the 1993 IEEE Signal Processing Society Senior Paper Award for the Speech Processing area. He received the IEEE Transactions on Multimedia Best Paper Award in 2010 and the IEEE Technical Committee on Wireless Communications Recognition Award for contributions in the area of Wireless Communications Systems and Networks in 2009.

Inhalt

Background and Overview.- Entropy and Mutual Information.- Differential Entropy, Entropy Rate, and Maximum Entropy.- Typical Sequences and The AEP.- Markov Chains and Cascaded Systems.- Hypothesis Testing, Estimation, Information, and Sufficient Statistics.- Information Theoretic Quantities and Learning.- Estimation and Entropy Power.- Time Series Analyses.- Information Bottleneck Principle.- Channel Capacity.- Rate Distortion Theory.