Information Theory, Coding and Compression

Bayesian estimation. Modelling and analysis of components in a generic communication system. Mathematical definitions of information content and channel capacity. Principles for optimal information transfer across various types of channels. Data compression. Principles and methods for practical digital representations. Practical channel coding. Performance assessment relative to information theoretic limits. Belief propagation.

The course shall give students a profund understanding of what information is made of, how it can be represented, stored or transmitted with as few loss or error as possible.

The course shall enable students to

Knowledge:
- understand the basic principles of information theory with respect to source and channel coding
- know some practical algorithms of information theory such as belief propagation and the sum-product algorithm
- know the pros and cons of various practical algorithms

General competence:
- understand information as a measure of probability

Abilities:
- select an appropriate algorithm for a given task
- find out which amount of information can be stored or transmitted in a given task
- calculate a-posteriori probabilities using Bayes' law

Lectures and exercises.