Elements of Information Theory, Second Edition, covers the standard topics of information theory, such as entropy, data compression, channel capacity, rate distortion, multi-user theory and hypothesis testing. It presents applications to communications, statistics, complexity theory, and investment. Chapters 1-9 cover the asymptotic equipartition property, data compression, and channel capacity culminating in the capacity of the Gaussian channel. Chapters 10-17 include rate distortion, the method of types, Kolmogorov complexity, network information theory, universal source coding, and portfolio theory.The first edition of this book is the most successful book on information theory on the market today. Adoptions have remained strong since the book's publication in 1991.

Zusatztext "This book is recommended reading! both as a textbook and as a reference." (Computing Reviews.com! December 28! 2006) Informationen zum Autor THOMAS M. COVER, PHD, is Professor in the departments of electrical engineering and statistics, Stanford University. A recipient of the 1991 IEEE Claude E. Shannon Award, Dr. Cover is a past president of the IEEE Information Theory Society, a Fellow of the IEEE and the Institute of Mathematical Statistics, and a member of the National Academy of Engineering and the American Academy of Arts and Science. He has authored more than 100 technical papers and is coeditor of Open Problems in Communication and Computation. JOY A. THOMAS, PHD, is the Chief Scientist at Stratify, Inc., a Silicon Valley start-up specializing in organizing unstructured information. After receiving his PhD at Stanford, Dr. Thomas spent more than nine years at the IBM T. J. Watson Research Center in Yorktown Heights, New York. Dr. Thomas is a recipient of the IEEE Charles LeGeyt Fortescue Fellowship. Klappentext THE LATEST EDITION OF THIS CLASSIC IS UPDATED WITH NEW PROBLEM SETS AND MATERIAL The Second Edition of this fundamental textbook maintains the book's tradition of clear, thought-provoking instruction. Readers are provided once again with an instructive mix of mathematics, physics, statistics, and information theory. All the essential topics in information theory are covered in detail, including entropy, data compression, channel capacity, rate distortion, network information theory, and hypothesis testing. The authors provide readers with a solid understanding of the underlying theory and applications. Problem sets and a telegraphic summary at the end of each chapter further assist readers. The historical notes that follow each chapter recap the main points. The Second Edition features: Chapters reorganized to improve teaching 200 new problems New material on source coding, portfolio theory, and feedback capacity Updated references Now current and enhanced, the Second Edition of Elements of Information Theory remains the ideal textbook for upper-level undergraduate and graduate courses in electrical engineering, statistics, and telecommunications. Zusammenfassung The latest edition of this classic is updated with new problem sets and material The Second Edition of this fundamental textbook maintains the book's tradition of clear, thought-provoking instruction. Readers are provided once again with an instructive mix of mathematics, physics, statistics, and information theory. Inhaltsverzeichnis Contents v Preface to the Second Edition xv Preface to the First Edition xvii Acknowledgments for the Second Edition xxi Acknowledgments for the First Edition xxiii 1 Introduction and Preview 1 1.1 Preview of the Book 5 2 Entropy, Relative Entropy, and Mutual Information 13 2.1 Entropy 13 2.2 Joint Entropy and Conditional Entropy 16 2.3 Relative Entropy and Mutual Information 19 2.4 Relationship Between Entropy and Mutual Information 20 2.5 Chain Rules for Entropy, Relative Entropy, and Mutual Information 22 2.6 Jensen's Inequality and Its Consequences 25 2.7 Log Sum Inequality and Its Applications 30 2.8 Data-Processing Inequality 34 2.9 Sufficient Statistics 35 2.10 Fano's Inequality 37 Summary 41 Problems 43 Historical Notes 54 3 Asymptotic Equipartition Property 57 3.1 Asymptotic Equipartition Property Theorem 58 3.2 Consequences of the AEP: Data Compression 60 3.3 High-Probability Sets and the Typical Set 62 Summary 64 Problems 64 Historical Notes 69 4 Entropy Rates of a Stochastic Process 71 4.1 Markov Chains 71 4.2 Entropy Rate 74THOMAS M. COVER, PHD, is Professor in the departments of electrical engineering and statistics, Stanford University. A recipient of the 1991 IEEE Claude E. Shannon Award, Dr. Cover is a past president of the IEEE Information Theory Society, a Fellow of the IEEE and the Institute of Mathematical Statistics, and a member of the National Academy of Engineering and the American Academy of Arts and Science. He has authored more than 100 technical papers and is coeditor of Open Problems in Communication and Computation. JOY A. THOMAS, PHD, is the Chief Scientist at Stratify, Inc., a Silicon Valley start-up specializing in organizing unstructured information. After receiving his PhD at Stanford, Dr. Thomas spent more than nine years at the IBM T. J. Watson Research Center in Yorktown Heights, New York. Dr. Thomas is a recipient of the IEEE Charles LeGeyt Fortescue Fellowship.

Klappentext

The latest edition of this classic is updated with new problem sets and material

The Second Edition of this fundamental textbook maintains the book's tradition of clear, thought-provoking instruction. Readers are provided once again with an instructive mix of mathematics, physics, statistics, and information theory.

All the essential topics in information theory are covered in detail, including entropy, data compression, channel capacity, rate distortion, network information theory, and hypothesis testing. The authors provide readers with a solid understanding of the underlying theory and applications. Problem sets and a telegraphic summary at the end of each chapter further assist readers. The historical notes that follow each chapter recap the main points.

The Second Edition features:

• Chapters reorganized to improve teaching
• 200 new problems
• New material on source coding, portfolio theory, and feedback capacity

• Updated references
  Now current and enhanced, the Second Edition of Elements of Information Theory remains the ideal textbook for upper-level undergraduate and graduate courses in electrical engineering, statistics, and telecommunications.

  Inhalt

  Contents v

  Preface to the Second Edition xv

  Preface to the First Edition xvii

  Acknowledgments for the Second Edition xxi

  Acknowledgments for the First Edition xxiii

  1 Introduction and Preview 1

  1.1 Preview of the Book 5

  2 Entropy, Relative Entropy, and Mutual Information 13

  2.1 Entropy 13

  2.2 Joint Entropy and Conditional Entropy 16

  2.3 Relative Entropy and Mutual Information 19

  2.4 Relationship Between Entropy and Mutual Information 20

  2.5 Chain Rules for Entropy, Relative Entropy, and Mutual Information 22

  2.6 Jensen's Inequality and Its Consequences 25

  2.7 Log Sum Inequality and Its Applications 30

  2.8 Data-Processing Inequality 34

  2.9 Sufficient Statistics 35

  2.10 Fano's Inequality 37

  Summary 41

  Problems 43

  Historical Notes 54

  3 Asymptotic Equipartition Property 57

  3.1 Asymptotic Equipartition Property Theorem 58

  3.2 Consequences of the AEP: Data Compression 60

  3.3 High-Probability Sets and the Typical Set 62

  Summary 64

  Problems 64

  Historical Notes 69

  4 Entropy Rates of a Stochastic Process 71

  4.1 Markov Chains 71

  4.2 Entropy Rate 74

  4.3 Example: Entropy Rate of a Random Walk on a Weighted Graph 78

  4.4 Second Law of Thermodynamics 81

  4.5 Functions of Markov Chains 84

  Summary 87

  Problems 88

  Historical Notes 100

  5 Data Compression 103

  5.1 Examples of Codes 103

  5.2 Kraft Inequality 107

  5.3 Optimal Codes 110

  5.4 Bounds on …