CS5105 : Fundamentals of Generative AI and Large Language Models: Theory and Practice (Autumn 2026)

Details of CS5105 (Autumn 2026)

Level: 5

Type: Theory

Credits: 4.0

Course Code	Course Name	Instructor(s)
CS5105	Fundamentals of Generative AI and Large Language Models: Theory and Practice	Saptarshi Pyne

Preamble
Preamble: This course is offered by NPTEL. The instructor listed here is the coordinator. The students have to register separately for the exam and write the proctored exam conducted by NPTEL in person at any of the designated exam centres. After obtaining the certificate from NPTEL, the students have to share the same with the instructor. Enrollment: A student has to enrol themself in the course on NPTEL and should also opt for the course in the Welearn portal. Following are the dates for enrolling in the NPTEL portal. Course start date: 20 Jul 2026 Course end date: 09 Oct 2026 Exam date (on NPTEL): 18 Oct 2026 Enrollment ends: ongoing, until 27 July, 2026 Exam registration ends: ongoing, until 14 August, 2026 Enroll here: https://onlinecourses.nptel.ac.in/e-learning/preview/noc26_cs95 Check here: https://nptel.ac.in/courses/106108002 Examination: The student has to register separately for the exam and write the proctored exam conducted by NPTEL in person at any of the designated exam centres. They need to register separately on NPTEL for the exam, pay a fee to NPTEL (Exam registration ends on 14 August, 2026) and write the proctored exam conducted by NPTEL in person at any of the designated exam centres. NPTEL will make the announcement regarding the commencement of registration for the examination. The online registration form must be filled, and the candidate must pay the certification exam fee. More details will be made available by NPTEL when the exam registration form is published on the NPTEL course website. If there are any changes, they will be mentioned then. Please check the form for more details on the cities where the exams will be held, the conditions you agree to when you fill the form etc. Note that, there will be no provision for a supplementary exam for NPTEL courses.

Preamble

Preamble:
This course is offered by NPTEL. The instructor listed here is the coordinator. The students have to register separately for the exam and write the proctored exam conducted by NPTEL in person at any of the designated exam centres. After obtaining the certificate from NPTEL, the students have to share the same with the instructor.

Enrollment: A student has to enrol themself in the course on NPTEL and should also opt for the course in the Welearn portal. Following are the dates for enrolling in the NPTEL portal.

Course start date: 20 Jul 2026
Course end date: 09 Oct 2026
Exam date (on NPTEL): 18 Oct 2026
Enrollment ends: ongoing, until 27 July, 2026
Exam registration ends: ongoing, until 14 August, 2026

Enroll here: https://onlinecourses.nptel.ac.in/e-learning/preview/noc26_cs95
Check here: https://nptel.ac.in/courses/106108002

Examination:

The student has to register separately for the exam and write the proctored exam conducted by NPTEL in person at any of the designated exam centres. They need to register separately on NPTEL for the exam, pay a fee to NPTEL (Exam registration ends on 14 August, 2026) and write the proctored exam conducted by NPTEL in person at any of the designated exam centres. NPTEL will make the announcement regarding the commencement of registration for the examination. The online registration form must be filled, and the candidate must pay the certification exam fee. More details will be made available by NPTEL when the exam registration form is published on the NPTEL course website. If there are any changes, they will be mentioned then. Please check the form for more details on the cities where the exams will be held, the conditions you agree to when you fill the form etc. Note that, there will be no provision for a supplementary exam for NPTEL courses.

Syllabus
Syllabus: Week 1: Fundamentals of Deep Learning for Generative AI What is Generative AI? Classical vs. Modern Generative Models Introduction to Neural Networks: Quick revision on MLP and optimizers Convolution operation, filters, feature extraction Pooling, padding, stride CNN architectures overview: LeNet, VGG, ResNet Feature maps & hierarchical representation Motivation for representation learning -> introduces autoencoders next week Hands-on Exercise: Build a CNN for a real-world dataset, implementation of transfer learning and building an ensemble model. Week 2: Autoencoders: The First Generative Model: What is an Autoencoder? Encode-Decoder architecture Bottleneck representation Undercomplete vs overcomplete AEs Training objective: reconstruction loss Denoising autoencoders Limitations of Autoencoders -> why VAEs are needed Hands-on Exercise: Implement a vanilla Autoencoder for the dataset, Add noise and train a Denoising Autoencoder (DAE), Visualize original vs reconstructed images and bottleneck vectors. Week 3: Variational Autoencoders (VAEs): Foundations: Motivation: Why VAEs instead of simple autoencoders? Latent variable models VAE architecture Reparameterization trick Evidence Lower Bound (ELBO) KL divergence intuition Sampling from latent space Hands-On Exercise: Implement a VAE for the dataset. Visualize 2-D latent space clusters using PCA/TSNE. Perform latent space interpolation to generate new digits. Week 4: Advanced VAEs & Practical Applications: Conditional VAE (CVAE) Beta-VAE + disentanglement Latent space interpolation Image generation using VAEs Strengths/limitations vs. GANs Why we need adversarial training -> leads to GANs Hands-On Exercise: Build a Conditional VAE for the dataset. Perform latent traversals to observe disentanglement. Compare VAE vs CVAE outputs for class-controlled generation. Week 5: Generative Adversarial Networks (GANs): Foundations What are GANs? Generator-Discriminator architecture Minimax objective GAN loss functions Training instability, mode collapse DCGAN architecture (the baseline GAN for images Hands-on Exercise: Implement DCGAN to generate medical images. Train and visualize generated samples every epoch. Demonstrate mode collapse by altering training settings. Week 6: Advanced GAN Variants & Applications: Conditional GAN (cGAN) CycleGAN (image-to-image translation) Pix2Pix StyleGAN + StyleGAN2 GAN evaluation metrics: IS, FID Real-world applications: Art generation Super-resolution Medical imaging augmentation Hands-On Exercise: Train a Conditional GAN for class-controlled image generation. Apply a pre-trained CycleGAN for style transfer (horses<->zebras or Monet painting). Compute FID score between generated and real samples. Week 7: Diffusion Models: Intuition & Forward Process: Why Diffusion Models? (advantages over GANs) Forward diffusion process (adding noise) Variance schedules Denoising intuition Concept of Markov chain Noise prediction objective (basic intuition) Hands-On Exercise: Simulate the forward diffusion process on real-time images. Visualize noise addition across timesteps. Implement a simple noise predictor model to understand training behavior. Week 8: Reverse Diffusion, UNet Architecture & Training: Reverse diffusion process Full DDPM training pipeline UNet architecture for denoising Loss functions Classifier-guided and classifier-free guidance Diffusion vs GANs vs VAEs (comparison) Hands-On Exercise: Implement a minimal DDPM pipeline for MNIST using a small UNet. Generate denoised samples from random noise. Experiment with different guidance weights and compare generation quality. Week 9: Sequence Models, NLP Basics & LSTMs: NLP Basics RNN and LSTM Concepts Why Transformers replaced LSTMs? Hands-On Exercise: Implement an LSTM for next-character prediction on a small text dataset (e.g., Shakespeare). Train a word-level LSTM for sentiment classification. Visualize hidden state dynamics and compare RNN vs LSTM accuracy. Week 10: Foundations of Large Language Models (LLMs) and Prompt Engineering basics: What is an LLM? Transformer architecture (encoder, decoder, decoder-only) Self-attention mechanism Tokenization: BPE, WordPiece Prompt engineering basics Hands-On Exercise: Implement a tiny Transformer for next-token prediction. Use HuggingFace to load a small LLM (DistilGPT-2) and generate text. Compare outputs using different sampling methods: greedy, top-k, nucleus. Week 11: Retrieval-Augmented Generation (RAG): Introduction to RAG Architecture of RAG Pipelines Retrieval Mechanisms: Generation Mechanisms with Retrieval In-context learning LoRA Hands-on Exercise: Build a complete RAG pipeline using Haystack / LangChain / LlamaIndex. Week 12: LLM Capabilities, Multimodal AI, Ethics: LLMs for Text generation LLMs for audio/video/multimodal tasks Size vs Performance Issues: Bias Fairness Hallucinations Safety frameworks Evaluation benchmarks Hands-On Exercise: Build a text-generation demo using an open-source LLM. Perform summarization and Q&A tasks with prompt engineering. Analyze hallucination behaviour using controlled prompts and document evidence.

Syllabus

Syllabus:

Week 1: Fundamentals of Deep Learning for Generative AI
What is Generative AI? Classical vs. Modern Generative Models
Introduction to Neural Networks: Quick revision on MLP and optimizers
Convolution operation, filters, feature extraction
Pooling, padding, stride
CNN architectures overview: LeNet, VGG, ResNet
Feature maps & hierarchical representation
Motivation for representation learning -> introduces autoencoders next week
Hands-on Exercise: Build a CNN for a real-world dataset, implementation of transfer learning and building an ensemble model.

Week 2: Autoencoders: The First Generative Model:
What is an Autoencoder?
Encode-Decoder architecture
Bottleneck representation
Undercomplete vs overcomplete AEs
Training objective: reconstruction loss
Denoising autoencoders
Limitations of Autoencoders -> why VAEs are needed
Hands-on Exercise: Implement a vanilla Autoencoder for the dataset, Add noise and train a Denoising Autoencoder (DAE), Visualize original vs reconstructed images and bottleneck vectors.

Week 3: Variational Autoencoders (VAEs): Foundations:
Motivation: Why VAEs instead of simple autoencoders?
Latent variable models
VAE architecture
Reparameterization trick
Evidence Lower Bound (ELBO)
KL divergence intuition
Sampling from latent space
Hands-On Exercise: Implement a VAE for the dataset. Visualize 2-D latent space clusters using PCA/TSNE. Perform latent space interpolation to generate new digits.

Week 4: Advanced VAEs & Practical Applications:
Conditional VAE (CVAE)
Beta-VAE + disentanglement
Latent space interpolation
Image generation using VAEs
Strengths/limitations vs. GANs
Why we need adversarial training -> leads to GANs
Hands-On Exercise: Build a Conditional VAE for the dataset. Perform latent traversals to observe disentanglement. Compare VAE vs CVAE outputs for class-controlled generation.

Week 5: Generative Adversarial Networks (GANs): Foundations
What are GANs?
Generator-Discriminator architecture
Minimax objective
GAN loss functions
Training instability, mode collapse
DCGAN architecture (the baseline GAN for images
Hands-on Exercise: Implement DCGAN to generate medical images. Train and visualize generated samples every epoch. Demonstrate mode collapse by altering training settings.

Week 6: Advanced GAN Variants & Applications:
Conditional GAN (cGAN)
CycleGAN (image-to-image translation)
Pix2Pix
StyleGAN + StyleGAN2
GAN evaluation metrics: IS, FID
Real-world applications:
Art generation
Super-resolution
Medical imaging augmentation
Hands-On Exercise: Train a Conditional GAN for class-controlled image generation. Apply a pre-trained CycleGAN for style transfer (horses<->zebras or Monet painting). Compute FID score between generated and real samples.

Week 7: Diffusion Models: Intuition & Forward Process:
Why Diffusion Models? (advantages over GANs)
Forward diffusion process (adding noise)
Variance schedules
Denoising intuition
Concept of Markov chain
Noise prediction objective (basic intuition)
Hands-On Exercise: Simulate the forward diffusion process on real-time images. Visualize noise addition across timesteps. Implement a simple noise predictor model to understand training behavior.

Week 8: Reverse Diffusion, UNet Architecture & Training:
Reverse diffusion process
Full DDPM training pipeline
UNet architecture for denoising
Loss functions
Classifier-guided and classifier-free guidance
Diffusion vs GANs vs VAEs (comparison)
Hands-On Exercise: Implement a minimal DDPM pipeline for MNIST using a small UNet. Generate denoised samples from random noise. Experiment with different guidance weights and compare generation quality.

Week 9: Sequence Models, NLP Basics & LSTMs:
NLP Basics
RNN and LSTM Concepts
Why Transformers replaced LSTMs?
Hands-On Exercise: Implement an LSTM for next-character prediction on a small text dataset (e.g., Shakespeare). Train a word-level LSTM for sentiment classification. Visualize hidden state dynamics and compare RNN vs LSTM accuracy.

Week 10: Foundations of Large Language Models (LLMs) and Prompt Engineering basics:
What is an LLM?
Transformer architecture (encoder, decoder, decoder-only)
Self-attention mechanism
Tokenization: BPE, WordPiece
Prompt engineering basics
Hands-On Exercise: Implement a tiny Transformer for next-token prediction. Use HuggingFace to load a small LLM (DistilGPT-2) and generate text. Compare outputs using different sampling methods: greedy, top-k, nucleus.

Week 11: Retrieval-Augmented Generation (RAG):
Introduction to RAG
Architecture of RAG Pipelines
Retrieval Mechanisms:
Generation Mechanisms with Retrieval
In-context learning
LoRA
Hands-on Exercise: Build a complete RAG pipeline using Haystack / LangChain / LlamaIndex.

Week 12: LLM Capabilities, Multimodal AI, Ethics:
LLMs for Text generation
LLMs for audio/video/multimodal tasks
Size vs Performance
Issues:
Bias
Fairness
Hallucinations
Safety frameworks
Evaluation benchmarks
Hands-On Exercise: Build a text-generation demo using an open-source LLM. Perform summarization and Q&A tasks with prompt engineering. Analyze hallucination behaviour using controlled prompts and document evidence.

Prerequisite
Students must have completed introductory courses in Programming and Machine Learning/Deep Learning. Knowledge of Python and basic mathematical concepts is necessary to follow the hands-on exercises.

References
References: 1. Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. Deep learning. MIT Press, 2016, ISBN : 9780262035613 2. Zhang, Aston, et al. "Dive into deep learning." Cambridge University Press, 2023,ISBN:9781009389433 3.CS231n:Deep Learning for Computer Vision, Stanford University 4.Practical Deep Learning, Fast.ai (https://course.fast.ai/)

Course Credit Options

Sl. No.	Programme	Semester No	Course Choice
1	IP	1	Not Allowed
2	IP	3	Not Allowed
3	MP	1	Not Allowed
4	MP	3	Not Allowed
5	MR	1	Not Allowed
6	MR	3	Not Allowed
7	MS	3	Not Allowed
8	MS	5	Not Allowed
9	MS	7	Not Allowed
10	MS	9	Elective
11	RS	1	Elective
12	RS	2	Elective

WeLearn :: Academic Portal of IISER Kolkata

Details of CS5105 (Autumn 2026)

Course Credit Options