On July 21, 2020, Jan Christopher Lewandowsky successfully defended his Ph.D. thesis The Information Bottleneck Method in Communications. The Ph.D. examination committee was formed by Prof. Dr. Herbert Werner (Institute of Control Systems) and the two reviewers Prof. Dr.-Ing. Gerhard Bauch and Prof. Dr.-Ing. Stephan ten Brink (University of Stuttgart).

Abstract

The receiver-sided high-precision signal processing for detection and channel decoding causes severe bottlenecks in modern communication receivers. Such bottlenecks concern the chip area, the power consumption and the processing delay of modern receivers. In this Ph.D. thesis, the Information Bottleneck method is used to design signal processing components with very coarse quantization and low complexity, but close-to-optimum performance.

The Information Bottleneck method is a generic information theoretical framework which aims for the compression of an observed random variable to a compressed random variable. The focal aim in designing this compression is to preserve relevant information. The method originates from machine learning and before this thesis only had a few practical applications in communications. The complexity of the receiver-sided baseband processing algorithms for demodulation and channel decoding causes a severe bottleneck in modern digital communication receivers. Their implementation complexity is mainly influenced by the bit width used to represent the signals processed in the hardware of the receiver and the arithmetical operations involved in the signal processing algorithms. Practical receiver implementations, therefore, have to be strongly quantized. Moreover, the involved operations have to be as simple as possible. The Information Bottleneck method aims to maximize the preserved relevant information for a given bit width hence motivating to apply the Information Bottleneck method to receiver design.

Figure 1: Illustration of the Information Bottleneck method

The first communication problem studied in the thesis is the design of channel output quantizers for various transmission channels with the Information Bottleneck method. The basic idea is to build quantizers that allow for a maximum possible preservation of relevant information for a given bit width. In the state-of-the-art approaches to quantization, the construction of quantizers is considered from a signal processing perspective, meaning that, for example, the mean squared error is minimized under quantization such that the quantized signal approximates the continuous input signal using representation values. Information theory, however, tells that the output information of a quantizer is independent of such representation values, but only depends on the probability distributions involved. Therefore, the thesis develops the unconventional approach to represent the outputs of the quantizers only using unsigned integer indices. Such integers can be represented and processed very efficiently in the receiver hardware. However, the conventional signal processing algorithms for detection and channel decoding need representation values. The thesis, therefore, develops numerous novel algorithms for detection and channel decoding which can handle the plain quantization indices and achieve excellent performance with very coarse quantization. As a starting point, quantized message passing decoders for regular and irregular low-density parity-check (LDPC) codes are developed with the Information Bottleneck method. The resulting decoders pair very coarse quantization (typically using just four bits per message) with very simple decoding operations only based on lookups in static tables. Interestingly, however, they achieve performance very close to non-quantized belief propagation decoders.

Figure 2: Integer decoding with the Information Bottleneck method

The constructed decoders are also applied to transmission systems with higher order modulation. For this purpose, Information Bottleneck demodulators are developed and investigated. After a thorough study of the decoding of regular and irregular LDPC codes with the Information Bottleneck method, the thesis widens the focus also to other parts of the receiver-sided signal processing chain. Novel quantized channel estimation and detection algorithms are developed with the Information Bottleneck method. The resulting receivers have virtually no loss in comparison to state-of-the-art receivers with double precision signal processing in terms of the end-to-end bit error rate, despite the fact that they work strongly quantized. Finally, the thesis quantifies practical gains of the Information Bottleneck approach to signal processing by investigating practical decoder implementations on a fixed-point digital signal processor.