GenAI & Agentic AI Workshop · Day 1 · Session 1

How LLMs Work

What actually happens when you ask ChatGPT a question?

Press → to move · S for speaker notes · F for full screen

The elephant in the room 🐘

Layoffs. Slow fresher hiring. "AI will replace programmers."

Let's be honest about what's happening, and what you can do about it.

Why are there layoffs?

📈 Over-hiring correction

Hired fast in 2020–22, cut back when demand cooled

💸 Cost pressure

Clients cut "nice to have" projects first. IT services feel it directly

🧠 Money moving to AI

GPUs, data centres, AI teams. Other teams shrink to pay for it

🤖 Routine tasks automated

Boilerplate code, simple tests, basic support. Often entry-level work

AI replaces tasks, not whole jobs. The work is changing shape, and AI builders are in demand.

What to improve

🧱 Fundamentals

DSA, DBMS/SQL, OS, networks. You can't judge AI's code without them

🧩 Problem solving

Deciding what to build and how it fits together

🔍 Reviewing code

Spot bugs and security holes in AI output

🗣️ Communication

Prompts, requirements, docs, interviews

🚀 Ownership

Finish and ship. One deployed project > ten tutorials

🏥 Domain knowledge

Finance, health, education... AI is generic, you aren't

What to learn: a roadmap

1. Foundations

Python, Git, SQL, APIs, JSON

→

2. Use AI daily

Chat + coding assistants

→

3. Build with LLMs

Prompts, APIs, RAG
Day 1

→

4. Build agents

Tools, MCP, multi-agent
Day 2

→

5. Production

Cloud, Docker, evals, cost

Goal: leave with a working AI agent on your GitHub.

Quick poll ✋

  • Who used ChatGPT, Gemini or Claude this week?
  • What did you use it for? Assignments? Code? Resume?
  • Has it ever given you a wrong answer confidently?

Today: how does it "know" things, and why does it sometimes get them wrong?

Today's plan

  1. AI → ML → Deep Learning
  2. Transformers & attention
  3. LLM = next-word predictor
  4. Tokens & embeddings
  1. Context window
  2. Temperature
  3. Hallucination
  4. Open vs closed models

Notes with all examples and code: see the Session 1 notes page.

AI, ML, DL, LLM: one inside the other

Artificial IntelligenceAny "smart" machine, including old rule-based systems
Machine LearningLearns patterns from examples
Deep LearningNeural networks with many layers
Generative AI / LLMsCreate new text, images, code

Rules vs examples

Traditional programming

Rules + data → answers

if "lottery" in email: spam

Spammer writes "l0ttery" → rule fails 😅

Machine learning

Data + answers → rules

Show 10,000 emails labelled spam / not spam

Model learns the patterns itself ✅

Campus example: past students' CGPA, internships, projects + "placed or not" → model predicts placement chance for a new student.

Deep learning: layers that learn

A neural network = layers of simple maths units. "Deep" = many layers.

Layer 1

Edges & lines

→

Middle layers

Eyes, nose, mouth

→

Last layer

"This is your face"

📱 Face unlock: nobody programmed "what an eye looks like". It learned from millions of images.

2017: the Transformer

Google paper: "Attention Is All You Need". The T in GPT = Transformer.

  • Old models: read one word at a time, forget the start of long sentences
  • Transformer: looks at all words at once
  • Attention: works out which words matter for each other

Attention in action

I went to the bank to deposit my scholarship money. → 🏦

We sat on the river bank and watched the sunset. → 🏞️

The dog didn't cross the road because it was too tired. → it = dog

The dog didn't cross the road because it was too busy. → it = road

An LLM is a next-word predictor

"Virat Kohli is famous for playing ___"

cricket92%
for3%
in2%
…

Pick a word → add it to the text → predict again → repeat. That's how a whole answer gets written.

The whole loop, in one picture

Text to tokens to embeddings to transformer to probabilities to pick one, then repeat

Key idea

An LLM doesn't look up answers.
It generates the most likely next piece of text,
one token at a time.

"Large" = billions of parameters: the numbers that store what it learned.

From predictor to assistant

1. Pre-training

Read a huge chunk of the internet. Learn to predict the next word.

📚 Reading a whole library

→

2. Instruction tuning

Learn from question → good answer examples.

📝 Practising exam answers

→

3. Human feedback

Humans rank answers. Model learns to be helpful & safe.

🧑‍🏫 A mentor's feedback

Tokens: how LLMs read

Not words, not letters: tokens (chunks of text).

"I love unbelievable biryani!" (illustrative split)

I love unbelievable biryani!

Rule of thumb: 1 token ≈ 4 characters ≈ ¾ word  ·  100 tokens ≈ 75 words

Why should you care about tokens?

💰 Cost

APIs charge per token, input + output.

📏 Limits

Each model has a max number of tokens.

🇮🇳 Indian languages

Hindi / Kannada / Tamil often use several times more tokens than English.

🍓 Strange mistakes

"How many r's in strawberry?" Older models said 2. They see strawberry, not letters.

Embeddings: meaning as numbers

samosadosachaibiryani placementjobrecruitmentinterview cricketbatwicketIPL

Text → list of numbers (a vector)

Similar meaning → close together

king − man + woman ≈ queen

Search by meaning, not keywords

Student asks: "How do I apply for placements?"

FAQ sentenceSimilarity*
What is the process for campus recruitment?0.82 ✅
Steps to register with the training & placement cell0.78 ✅
Placement of furniture in the new hostel block0.21 ❌

*Illustrative scores. This is the core of RAG, which we build in Session 4.

Context window: the model's desk

📖 Open-book exam, but your desk fits only N pages. Add more, and the oldest pages fall off.

  • Max tokens the model sees at once: instructions + chat history + documents + answer
  • Ranges from a few thousand to hundreds of thousands (even 1M) of tokens
  • Long chat → old messages dropped → bot "forgets"
  • No memory between calls. Apps re-send the whole chat every time (Session 3)

Temperature: safe or creative?

Which temperature when?

TaskTemperature
Answer from syllabus, extract JSON, write codeLow (0 – 0.3)
General chat, explanationsMedium (0.5 – 0.8)
Tech fest names, poems, brainstormingHigh (0.9 – 1.2)

Related setting: top-p. Usually tune one, not both.

Hallucination 🤯

A confident, fluent answer that is false or made up.

Prompt: "Summarise the 2019 paper 'Quantum Biryani Optimization' by Dr. R. Sharma, IIT Bombay."

This paper doesn't exist. A model may still write a detailed summary, because that's what summaries look like.

Also common: fake citations, wrong dates, made-up code libraries.

Why it happens, and how to reduce it

Why

  • Predicts plausible text, not true text
  • No built-in fact database
  • Knowledge cutoff date
  • Rewarded for answering, not for "I don't know"

Reduce it

  • Give it the source (RAG, Session 4)
  • Say: "If not in the text, say you don't know"
  • Ask for citations, then check them
  • Low temperature, tools, human review

Open vs closed models

🔒 Closed

GPT (OpenAI) · Claude (Anthropic) · Gemini (Google)

  • Use via app / API only
  • ✅ Usually most capable, no hardware
  • ❌ Pay per token, data goes to provider

🔓 Open-weight

Llama (Meta) · Mistral · Gemma · Qwen · DeepSeek

  • Download & run yourself (Ollama, own server, Groq)
  • ✅ Control, privacy, customisable
  • ❌ Need GPUs & skills, licences vary

How to choose a model

🎯 Quality

Test on 10–20 of your examples

💰 Cost

Requests × tokens × price

⚡ Speed

Live chat or overnight job?

🔐 Privacy

Can the data leave?

🗣️ Language

Hindi, Kannada, Tamil?

📜 Licence

Commercial use allowed?

💡 Start with the best model to prove it works, then try cheaper ones.

You decide: which model? 🗳️

ScenarioPick
24-hour hackathon prototypeClosed API / free tier
Hospital records, data can't leave the buildingOpen-weight, self-hosted
Classify 10 lakh reviews a daySmall, cheap, fast model
Support chatbot in KannadaTest several on real Kannada questions

Recap: 6 things to remember

  1. ML learns rules from examples
  2. Transformers use attention to read words in context
  3. LLMs predict the next token, again and again
  4. Tokens = cost & limits · Embeddings = meaning as numbers
  5. Context window = short-term memory · Temperature = creativity dial
  6. LLMs hallucinate: ground them with sources

Quick quiz 🧠

  1. Roughly how many tokens is a 750-word essay? ≈ 1,000
  2. Why does a chatbot forget the start of a long chat? Context window full
  3. Extracting marks into JSON: high or low temperature? Low
  4. Name one way to reduce hallucination. Give sources / RAG

Up next · Session 2

Prompt Engineering

Same model, better answers: zero-shot, few-shot, chain-of-thought, JSON output

Questions? 🙋