- Methods of performing localization (With the example of the self-driving car)
- Various sensors are used for localization (LiDAR, RADAR, GNSS, INS, wheel encoders)
- Sensing models for sensors used in practice that increase the presence of uncertain results in imperfect models
- Downloading Python and Jupyter notebook
- Numpy and matplotlib
- Home
- Skill Lync
- Courses
- Post Graduate Program in Sensor Fusion
Online
32 Weeks
Quick facts
particular | details | |
---|---|---|
Medium of instructions
English
|
Mode of learning
Self study
|
Mode of Delivery
Video and Text Based
|
Course and certificate fees
certificate availability
No
The syllabus
Course 1: Localization, Mapping and SLAM using Python
Week 01 - Introduction to Localization
Week 02 - Probability Theory Refresher
- Concepts such as random variable, random vectors, density function, joint density, marginal density, conditional independence
- Probability Density Function (PDF), gaussian, multivariate gaussian
- Variance and covariance
- Bayes rule with multiple variables
- Random variables in robotics state estimation
Week 03 - Probabilistic modelling & Bayesian Filtering
- Probabilistic models for perception and state transition
- Fitting model from Data, Maximum Likelihood Estimate (MLE)
- Bayesian filter, Markov assumption
- Kalman filters and Particle filters
Week 04 - Kalman Filter
- Kalman filter and its derivation from Bayes filter
- Kalman Gain (with an example)
- Kalman filter assumptions and optimality
Week 05 - Extended Kalman Filter and Unscented Kalman Filter
- Limitations of Kalman filter
- Variants that overcome the limitations
- Jacobians in Extended Kalman filter
Week 06 - Particle Filter (aka Monte Carlo Localization)
- Map-based localization
- Derivation of particle filter from the Bayes filter.
- PF handling non-linearity.
- Properties of particle filter
Week 07 - Multi Sensor Fusion
- Loosely coupled and tightly coupled techniques in Extended Kalman filtering
- Extension of state vector to accept multiple inputs
Week 08 - Introduction to Mapping and SLAM
- Map generation process from the perspective of self-driving car
- SLAM problem (similar to chicken and egg problem)
- The simplest implementation of SLAM – EFF SLAM and loop closure
Week 09 - Graph SLAM
- Difference between offline SLAM and online SLAM Motivate Graph-based modelling of a SLAM problem
- The mathematical formulation of Graph SLAM and how it is used offline to generate a map of the environment
Week 10 - FastSLAM
- Particle filter-based SLAM
- Mathematical derivation and comparison with three techniques of SLAM discussed
Week 11 - Other Implementations for SLAM
- Factor graph and pose graph formulations of SLAM problem
- Use of camera as another sensing source in SLAM [Optional]
- An example of how to factor graph is implemented on a drone fitted with a downward-facing camera [Optional]
- Robot Operating System – what it is and important concepts (Publisher, Subscriber, topics, message, and more)
Week 12 - ROS Extra Lecture
- Cmake & catkin system,
- Advance concepts such as action server and transform tree
- Importance of open source
- Monte Carlo Localization as a system that is deployed in simulation
Course 2: Radar Sensor Processing
Week 01- Introduction to ADAS
- Introduction to ADAS
- Different types of ADAS
- Applications of ADAS
Week 02- Autonomous Driving (AD)
- Introduction of Autonomous Driving (AD)
- Levels of Autonomous Driving
- The overall architecture of AD
- How Computer Vision is used in AD?
- What’s Deep Learning's role in AD?
Week 03- Understanding of Basic sensors used in AD
- Sensors roles in Autonomous driving
- How different sensors work in AD:
- Camera Sensors
- Radar
- LiDAR
- Sensor Fusion - Camera, LiDAR, Radar
- How safety is achieved with multiple sensors
Week 04- Introduction to RADAR sensor
- What is Radar?
- Automotive Radar
- How Radar Sensor Looks
- Radar sensor on Vehicle
Week 05- Radar Signal Processing
- Introduction
- Components of Radar Signal processing
- Range Equation
- FMCW
- FMCW – Terms and Definitions
- Measurement of Range (Distance)
- Measurement of Doppler Velocity
- Measurement of Angle/Angle of Arrival
- Measurement of RCS
Week 06- Advance Radar Signal Processing
- Introduction
- Range FFT and Doppler FFT
- Angle FFT and RD Map
- Clutter Removal and CFAR
- Final Detection List
Week 07- Radar Technical details
- Introduction
- Pulse Repetition Frequency
- Duty Cycle
- Dwell Time/ Hits per Scan
- The Radar Equation
- Free-Space Path Loss
- Derivation of the Radar Equation
- Radar Cross-Section
- Losses
Week 08- Radar Devices and roles
- Classification of Radar System
- Radar Devices and their functionalities
- Role of Radar Sensors
- Importance of Radar
- Automotive Radar
- Radar Signal Processing in automotive systems
Week 09- Radar Data Processing
- Introduction
- Data processing
Week 10- Environment Setup
- Setting up things/ pre-requirements for coding and setting up the environment.
- Getting Git Ready
- Git basics
- Creating repo
- Hands-on the Git setup.
- Installation of Compilers
- Device Setup
Week 11- RADAR+ AI
- Deep Learning: PointNet and PointPillars
- Discussion of the Network and how to change Lidar data for the network.
Course 3: Introduction to Camera Systems Using C++
Week 1 - Camera Construction
- Introduction to Geometrical Construction
- Introduction to Optical Construction
- Introduction to Camera Types
- Camera Sensor Types – CCD, CMOS
- Camera Sensor Types – RGGB, RCCB, RCCC
- Different Lens Types – Normal vs Fisheye
- Optical Parameters – Exposure Time, Shutter, White Balance, Gain
Week 2 - Camera Models
- Different Camera Models
- Pin hole model, Perspective model, fisheye model
- Lens Distortion – Barrel /Radial, Pin Cushion
- Depth Of Field , Field of View
- Effects on changing aperture
Week 3 - Camera Calibration
- Camera Calibration
- Introduction to Camera Parameters
- Calibration Techniques
- Calibration for Intrinsic vs Extrinsic
- Image Undistortion
Week 4 - Projective Geometry
- Introduction to Projective Geometry
- What is Lost / Preserved ?
- Vanishing Lines & Points
- Dimensionality Reduction
- World to Image Projection
- Orthographic Projection
Week 5 - Stereo Vision
- Introduction To Stereo Vision
- Basic Idea of Stereo
- Epipolar Geometry
- Image rectification
- Stereo Correspondence
- Disparity Maps
- Depth Maps
Week 6 -Camera Systems
- Low FOV Long range cameras
- Stereo Camera
- FLIR Camera
- Fisheye Camera – Continental
- Camera Parameters
- Different Uses for each of them
Week 7 - Image Pre-Processing
- Image Color Spaces
- Color Space conversions (RAW -> RGB, RGB-> GRAYSCALE, RGB->YUV , …)
- Image Digitization, Sampling, Quantization
- Image Interpolation, Extrapolation
- Image Normalization
- Image Noise – Salt and Pepper noise, Gaussian Noise , Impulse Noise
- Image Erosion/Dilution
Week 8 - Image Processing -1 (Transformations)
- Basic Transformations and Filtering
- Domain Transformations
- Noise Reduction
- Filtering as Cross Correlation
- Convolution
Week 9 - Image Processing -2
- Basic Image Filtering and Detection techniques
- Corners Detection
- Edge Detection
- Contour Detection
- Image Thresholding Histogram
- Histogram Equalization
Week 10 - Image Processing -3
- Features and Image Matching
- Image Features, Invariant Features (Geometrical, Photometric Invariance)
- Image Descriptors
- HOG
- SIFT
- SURF
- Image Stitching
Week 11 - Image Processing - 4
- Introduction to Structure from Motion (SFM)
- Epipolar Constraint and Essential Matrix
- 3d Reconstruction
- Bundle Adjustment
- SVD approach to SFM
- SLAM example
Week 12- Introduction to Embedded Systems
- Camera Interfaces. Ex : GMSL, LVDS
- Communication Protocol – I2C
- Camera Initialization Sequence
- Automated Exposure Gain (AEG) Control
- Vision Processing Units (VPU)
- Graphic Processing units
Course 4: Math behind Machine Learning & Artificial Intelligence using Python
Week - 01 Basic concepts
- Sets
- Subsets
- Power set
- Venn Diagrams
- Trigonometric Functions
- Straight lines
- A.M, G.M, and H.M
- Concepts of Vectors
Week - 02 Permutation & Combinations
- Introduction to permutation and combinations
- Basics of permutation and combinations
- Fundamental principle of counting
- Permutations
- Combinations
Week - 03 Statistics - I
- First business moment
- Second business moment
- Third business moment
- Fourth business moment
Week - 04 Probability
- Introduction to probability
- Random experiments
- Conditional probability
- Joint probability
Week - 05 Statistics – II
- Z Scores
- Confidence interval
- Correlation
- Covariance
Week - 06 Probability - II
- Introduction to probability - II
- Uniform Distribution
- Normal Distribution
- Binomial Distribution
- Poisson Distribution
Week - 07 Likelihood (for Logistic regression)
- Introduction to likelihood statistics
- Odds
- Log odds
- Maximum likelihood vs probability
- Logistic regression
Week - 08 Gradient descent (for Linear & Logistic regression)
- Loss function
- Cost function
- The gradient descent for linear regression
- The gradient descent for logistic regression
Week - 09 Linear Algebra (for PCA)
- Matrices
- Types of matrices
- Operation on matrices
- Eigen values
- Eigen vectors
Week - 10 Derivatives (for Neural network)
- Derivatives
- Intuitive ideas of derivatives
- Increasing & decreasing function
Week - 11 Backpropagation (for Deep learning)
- Chain rule
- Maxima & minima
- Back propagation
- The cost function for deep learning
Week - 12 Python
- Basics of Python
- If else
- For loop
- Data types