💪
3 Week Bootcamp: Building Realtime LLM Application
  • Introduction
    • Timelines and Structure
    • Course Syllabus
    • Meet your Instructors
    • Action Items
  • Basics of LLM
    • What is Generative AI?
    • What is a Large Language Model?
    • Advantages and Applications of Large Language Models
    • Bonus Resource: Multimodal LLMs and Google Gemini
  • Word Vectors Simplified
    • What is a Word Vector
    • Word Vector Relationships
    • Role of Context in LLMs
    • Transforming Vectors into LLM Responses
      • Neural Networks and Transformers (Bonus Module)
      • Attention and Transformers (Bonus Module)
      • Multi-Head Attention, Transformers Architecture, and Further Reads (Bonus Module)
    • Graded Quiz 1
  • Prompt Engineering
    • What is Prompt Engineering
    • Prompt Engineering and In-context Learning
    • Best Practices to Follow in Prompt Engineering
    • Token Limits in Prompts
    • Ungraded Prompt Engineering Excercise
      • Story for the Excercise: The eSports Enigma
      • Your Task
  • Retrieval Augmented Generation and LLM Architecture
    • What is Retrieval Augmented Generation (RAG)?
    • Primer to RAG: Pre-Trained and Fine-Tuned LLMs
    • In-Context Learning
    • High-level LLM Architecture Components for In-context Learning
    • Diving Deeper: LLM Architecture Components
    • LLM Architecture Diagram and Various Steps
    • RAG versus Fine-Tuning and Prompt Engineering
    • Versatility and Efficiency in Retrieval-Augmented Generation (RAG)
    • Key Benefits of RAG for Enterprise-Grade LLM Applications
    • Similarity Search in Vectors (Bonus Module)
    • Using kNN and LSH to Enhance Similarity Search in Vector Embeddings (Bonus Module)
    • Graded Quiz 2
  • Hands-on Development
    • Prerequisites
    • Dropbox Retrieval App in 15 Minutes
      • Building the app without Dockerization
      • Understanding Docker
      • Building the Dockerized App
    • Amazon Discounts App
      • How the Project Works
      • Repository Walkthrough
    • How to Run 'Examples'
  • Bonus Resource: Recorded Interactions from the Archives
  • Bootcamp Keynote Session on Vision Transformers
  • Final Project + Giveaways
    • Prizes and Giveaways
    • Tracks for Submission
    • Final Submission
Powered by GitBook
On this page
  • What you'll learn from this session
  • More about the keynote speaker

Bootcamp Keynote Session on Vision Transformers

PreviousBonus Resource: Recorded Interactions from the ArchivesNextFinal Project + Giveaways

Last updated 1 year ago

This live session features a deep dive into recent advances in computer vision: from Convolutional Nets to Spectral Transformers, with , Senior Director and ML Research Leader at Microsoft.

This session sheds light on the latest in computer vision, vision transformers, and their role in Large Language Models (LLMs).

Towards the end, our host Mudit Srivastava from Pathway interacts with Vijay as he addresses curated questions from your fellow learners who had joined live.

What you'll learn from this session

  • The transition from traditional convolutional networks to pre-trained transformers in computer vision.

  • The synergy between these advanced transformers and LLMs leading to enhanced image classification and other tasks.

  • Insight into Scattering Vision Transformers (SVT), detailing their development, technical aspects, and performance.

  • Demonstration of SVT's leading performance in tasks like image classification (ImageNet dataset) and instance segmentation (MSCoco dataset).

More about the keynote speaker

Dr. Vijay Srinivas Agneeswaran, Sr. Director and ML Research Leader at Microsoft, brings over two decades of expertise in AI, machine learning, and data science. Holding a Ph.D. from IIT Madras and a postdoctoral from EPFL, his specializations include computer vision, efficient transformers, and large language models. At Microsoft, he has led pioneering research in AI for C+AI data and developed spectral transformers for computer vision, showcased at NeurIPS 2023. He is a champion of responsible AI, ensuring compliance for nearly 50 AI models, and has led teams in organizations like Walmart Global Tech, Oracle, and Cognizant, attesting to his significant industry impact. Dr. Agneeswaran also holds five US patents and is a prolific contributor to tech conferences and publications.

Dr. Vijay Srinivas Agneeswaran