M-ary PSK (Phase Shift Keying) - Part- II, Optimal Decision Rule, Nearest Neighbor Criterion and Approximate Probability of Error
Introduction to Information Theory, Relevance of Information Theory and Characterization of Information
Definition of Entropy, Average of Information / Uncertainity of source and Properties of Entropy
Entropy Example- Binary Source Maximum and Minimum Entropy of Binary Source
Maximum Entropy of Source with M-ary Alphabet, Concave/Convex Functions and Jensens Inequality
Joint Entropy , Definition of Joint Entropy of Two Sources and Simple Examples for Joint Entropy Computation
Properties of Joint Entropy and Relation between Joint Entropy and Marginal Entropies
Conditional Entropy, Example of Conditional Entropy and Properties of Conditional Entropy
Mutual Information, Diagrammatic Representation and Properties of Mutual Information
Simple Example of Mutual Information and Practical Example of Mutual Information-Binary Symmetric Channel
Channel Capacity, Implications of Channel Capacity, Claude E. Shannon- Father of Information Theory and Example of Capacity of Binary Symmetric Channel
Differential Entropy and Example for Uniform Probability Density function
Differential Entropy of Gaussian Source and Insights
Joint Conditional/ Differential Entropies and Mutual Information
Capacity of Gaussian channel- Part I
Capacity of Gaussian Channel Part-II, Practical Implications and Maximum rate in bits\sec
Introduction to Source Coding and Data Compression, Variable Length codes and Unique Decodability
Uniquely Decodable Codes, Prefix-free code, Instantaneous Code and Average Code length
Binary Tree Representation of Code, Example and Kraft Inequality
Lower Bound on Average Code Length and Kullback-Leibler Divergence
Optimal Code length, Constrained Optimization and Morse Code Example
Approaching Lower Bound on Average code length and Block Coding
Huffman Code, Algorithm, Example and Average Code Length
Introduction to channel coding, Rate of Code, Repetition Code and Hamming Distance
Introduction to Convolutional Codes, Binary Field Arithmetic and Linear Codes
Example of Convolutional Code Output and Convolution Operation for Code generation
Matrix Representation of Convolutional Codes, Generator Matrix, Transform Domain Representation and Shift Register Architecture
State Diagram Representation of Convolutional Code, State transitions and Example of Code Generation using State transitions
Trellis Representation of Convolutional Code and Valid Code Words
Decoding of the Convolutional Code, Minimum Hamming distance and Maximum Likelihood Codeword Estimate
Principle of Decoding of Convolutional code
Viterbi Decoder for Maximum Likelihood Decoding of Convolutional Code Using Trellis Representation, Branch Metric Calculation, State Metric Calculation and Example