Probabilities and Random Signals

Short introduction to probability theory (Set theory, Probability Models, Definitions and axioms, Conditional probabilities, main theorems, independence) Random Variables – R.V. - Definition and basic concepts, - discrete R.V.: Probability Mass Functions, R.V. functions, Expected Value and Variance, Joint probability Mass Function of multiple discrete R.V., Conditionality, Independence, - continuous R.V.: Probability Density Function, Cumulative Distribution Function, normal R.V., Conditional probability density function by an event, multiple continuous R.V., Distributions of functions of R.V. Additional R.V. concepts (sums of independent R.V., convolution, more on conditional expected value and variance, Sums of a random number of independent R.V., covariance and correlation. Introduction to stochastic processes, Bernoulli and Poisson. Random vectors, multidimensional normal distributions, linear transforms of random vectors. Random signals: definition, interpretation, special cases. Expected Values and autocorrelation function of random signals. Stationarity. Power spectral Density. Response of linear, time invariant systems to random signals.

Objectives

Upon successful completion of the course, the student is expected to be able to understand and use all the basic concepts of the probabilities, the random variables, the distributions, the random vectors, stochastic processes and random signals, correlation functions and autocorrelation. Also the concepts of power spectral density, linear transforms of random vectors and linear filtering of random signals are also taught. The concepts of probabilities and statistics taught in the course are the necessary basis for many advanced courses in the fields of pattern recognition, machine learning, artificial intelligence, signal processing, speech, image and video processing, telecommunication systems, computer networks e.t.c. which are taught in the engineering schools and the schools of informatics. The course includes an extended set of MATLAB laboratory exercises which provide to the student an explicit understanding of how the theory is applied in practice.

Prerequisites

Required: Basic knowledge of Mathematics as the ones taught in the course: Mathematics I. Recommended: Basic knowledge of signals and systems.

Syllabus

D1. Applications of Probabilities and Statistics D2. Introduction to Probabilities. Basic concepts. Sample space, Algebra of sets, probability measure, Conditional probability, Basic probability rules. D3. Independence, Conditional independence, Bayes Networks. D4. Discrete random variables, definition, examples, probability mass function, distribution function D5-D6. Probability rules for random variables, independence of random variables, functions of random variables, Expected value D7-D8. Continuous random variables. Definition, Examples. Probability Density function (PDF), Distribution function, Probability rules for continuous random variables, Independence Functions of continuous random variables, Expected values. D9-D10. Combinatorics. Common distributions (Bernoulli, Binomial, Geometric, Poisson, Exponential, Uniform, Normal) D11-D12. Examples on distributions, Weak law of large numbers - Strong law of large numbers - Central limit theorem D13-D14. Random vectors D15-D16. Repeat on Random vectors. RVs and correlation/independence, Multivariate normal distribution, Examples D17-D18. Introduction to random signals, Expected values of Random Signals. Correspondence to random variable and random vector, Special types of random signals (Bernoulli, Poisson,Markov), Markov Chains and Markov Models D19. Stationarity of Random signals, Strict sense stationarity, Wide sense stationarity, Autocorrelation, Autocorrelation properties. D20-D21. Power spectral density, Properties of power spectral density, White noise, Low-pass and band-pass random signals, Power spectral density of discrete random signals, examples D22-D23. Random signals through linear systems, Response of discrete and continuous LTI systems. Autocorrelation of LTI systems, Fourier transform of the system impulse response, Input stationary signals, Linear transforms of stochastic processes, Power spectral density of the output, Examples