GenAI & Agentic AI Workshop
Day 1 · Session 1 · 1 hour

How LLMs Work

What happens inside ChatGPT, Gemini or Claude when you type a question? This session explains it in plain language. No heavy math.

Open the slides Jump to the quiz
What you will learn
  1. Introduction: layoffs, AI and what it means for you
  2. The big picture: AI → ML → Deep Learning → LLMs
  3. Transformers and attention
  4. An LLM is a next-word predictor
  5. Tokens
  6. Embeddings
  7. Context window
  8. Temperature
  9. Why LLMs hallucinate
  10. Open vs closed models, and how to choose
  11. Recap, quiz and glossary

0. Introduction: why this workshop, why now

You have probably seen the headlines: big tech companies and IT services firms cutting jobs, fresher hiring slowing down, and "AI will replace programmers" posts everywhere. Before we learn how LLMs work, let's be honest about what is happening and what you can do about it.

Why are there layoffs?

There is no single reason. Several things are happening at the same time:

  1. Correction after over-hiring. During 2020–22, demand for digital services exploded and companies hired very fast. When demand cooled, many of them cut back.
  2. Cost pressure. Higher interest rates and an uncertain economy mean companies and their clients cut "nice to have" projects first. Indian IT services feel this directly when overseas clients reduce spending.
  3. Money is moving to AI. Companies are spending heavily on GPUs, data centres and AI teams. To pay for it, they often shrink other teams. The budget didn't disappear; it moved.
  4. Routine tasks are being automated. AI tools now handle a lot of first-draft work: boilerplate code, simple test cases, basic support tickets, documentation, data entry. Roles made mostly of these tasks are hit hardest, and many of those are entry-level roles.
  5. Flatter teams. Many companies are removing layers of management and expecting smaller teams to deliver more, often with the help of AI tools.
The real picture AI mostly replaces tasks, not whole jobs. The work isn't vanishing; it is changing shape. A developer who uses AI well can now do the work that used to take a small team, and companies are hiring for that person. At the same time, demand for people who can build, deploy and evaluate AI systems keeps growing.
Example: the same task, two ways

Before: a fresher spends two days writing CRUD APIs and unit tests for a student-records module.

Now: an AI assistant writes the first draft in minutes. The valuable work is everything around it: understanding what the college actually needs, designing the data model, checking the AI's code for bugs and security issues, and deploying it reliably.

The person who can only type the code is easy to replace. The person who can do the rest is in demand.

What to improve

SkillWhy it matters more now
Fundamentals: DSA, OOP, DBMS/SQL, networking, operating systemsYou can't judge whether AI-written code is correct, fast or secure if you don't understand it yourself.
Problem solving and system designAI is good at writing a function. Deciding what to build and how the pieces fit together is still your job.
Reading and reviewing codeYou will read far more code than you write. Spotting bugs in AI output is a core skill.
Communication: writing, explaining, asking good questionsPrompting is communication. So are requirements, documentation and interviews.
Ownership: finishing and shipping thingsA deployed project that real people use beats ten unfinished tutorials.
Domain knowledge: finance, healthcare, education, manufacturing...AI is generic. Knowing a business problem deeply is what makes your solution useful.

What to learn

A practical path, roughly in order. This workshop covers the middle of it.

1. FoundationsPython, Git and GitHub, SQL, REST APIs and JSON, basic Linux.
2. Use AI tools dailyChatGPT, Gemini, Claude, coding assistants. Learn where they help and where they fail.
3. Build with LLMsPrompt engineering, calling LLM APIs, RAG with embeddings and vector databases. (Day 1)
4. Build agentsTool calling, MCP, agent frameworks, multi-agent systems. (Day 2)
5. Take it to productionCloud basics (AWS, Azure or GCP), Docker, evaluation, monitoring, cost, security.
Your goal for these two days Leave with one working AI project on your GitHub: a campus helpdesk agent you built yourself. Then improve it, deploy it and put it on your resume. That is worth more in an interview than any certificate.

1. The big picture

People use "AI", "ML", "deep learning" and "GenAI" as if they mean the same thing. They don't. Each one sits inside the one before it:

Artificial Intelligence (AI)Any machine that does something we'd call "smart". Includes old rule-based systems.
Machine Learning (ML)The machine learns patterns from examples instead of being given rules.
Deep Learning (DL)ML with large neural networks that have many layers.
Generative AI and LLMsDeep learning models that create new text, images, code or audio.

Traditional programming vs machine learning

Traditional programmingMachine learning
You give itRules + dataData + correct answers (examples)
It gives youAnswersThe rules (a "model")
Spam filter exampleif "lottery" in email: spam. Spammers write "l0ttery" and it breaks.Show it 10,000 emails labelled spam / not spam. It learns the patterns itself, including ones you never thought of.
Example: placement prediction Give an ML model past student records (CGPA, internships, projects, communication score) and whether each student got placed. It learns which factors matter and predicts the chance for a new student. Nobody wrote the rule "CGPA > 7.5 and 1 internship = likely placed". The model found it.

Deep learning

Deep learning uses neural networks: layers of simple maths units loosely inspired by brain neurons. "Deep" just means many layers. Each layer learns something a little more complex than the one before.

Example: face unlock on your phone Early layers spot edges and lines. Middle layers spot eyes, nose, mouth. Final layers recognise your face. No one programmed "what an eye looks like". The network learned it from millions of images.

2. Transformers and attention

In 2017, researchers at Google published a paper called "Attention Is All You Need". It introduced the transformer, the design behind almost every modern LLM. The "T" in GPT stands for Transformer.

Older models read a sentence one word at a time, left to right, and often forgot the start of long sentences. A transformer looks at all the words at once and works out which words matter for each other. This is called attention.

Example: the word "bank"

"I went to the bank to deposit my scholarship money." → attention links bank with deposit and money, so it means a financial bank.

"We sat on the river bank and watched the sunset." → attention links bank with river, so it means the side of a river.

Same word, different meaning, decided by the words around it.
Example: what does "it" mean? "The dog didn't cross the road because it was too tired." → it = the dog.
"The dog didn't cross the road because it was too busy." → it = the road.
You figured this out instantly. Attention is how a transformer does the same.

3. An LLM is a next-word predictor

A Large Language Model (LLM) is a huge transformer trained on a huge amount of text (books, websites, code, Wikipedia) to do one simple job: predict the next word.

Example Input: "Virat Kohli is famous for playing ___"
The model scores every possible next word: cricket 92%, for 3%, in 2%, ... and picks one. Then it adds that word to the input and predicts the next one again. Word by word, it writes a whole answer.
Key idea An LLM does not "look up" answers. It generates the most likely continuation of the text, one piece at a time. It's very good at this because it has seen so much text. Everything else in this workshop builds on this one idea.
Your text is split into tokens, turned into embeddings, processed by the transformer with attention, which outputs probabilities for the next token. One token is picked, added to the text, and the loop repeats.

"Large" refers to the number of parameters: the adjustable numbers inside the network that store what it learned. Modern LLMs have billions of them.

From predictor to helpful assistant

A model that only predicts the next word isn't a good chatbot yet. Ask it "What is the capital of France?" and it might continue with "What is the capital of Germany?" because that looks like a quiz list. So it is trained in stages:

1. Pre-trainingRead a huge part of the internet and learn to predict the next word. Like reading an entire library.
2. Instruction tuningLearn from thousands of example question → good answer pairs. Like learning to answer exam questions.
3. Human feedback (RLHF)Humans rank answers; the model learns to prefer helpful, honest, safe ones. Like a mentor giving feedback.

4. Tokens

LLMs don't read words or letters. They read tokens: small chunks of text. A token can be a whole word, part of a word, a space plus a word, or punctuation.

Example (illustrative split) "I love unbelievable biryani!" might become:
I love unbelievable biryani!
Common words are usually one token. Rare words get split into pieces. The exact split depends on each model's tokenizer.
Rule of thumb (English) 1 token ≈ 4 characters ≈ ¾ of a word. So 100 tokens ≈ 75 words, and a 2-page essay ≈ 1,000 tokens.

Why tokens matter

Try it Paste a sentence into OpenAI Tokenizer or Tiktokenizer. Try the same sentence in English and in Hindi and compare the token count.
# Count tokens in Python (run in Google Colab)
!pip install tiktoken
import tiktoken

enc = tiktoken.get_encoding("cl100k_base")
text = "I love unbelievable biryani!"
tokens = enc.encode(text)
print(len(tokens), "tokens")
print([enc.decode([t]) for t in tokens])

5. Embeddings

Computers work with numbers, not meaning. An embedding turns a word, sentence or document into a list of numbers (a vector) so that things with similar meaning end up close together.

samosadosachaibiryani Food placementjobrecruitmentinterview Jobs cricketbatwicketIPL Cricket
Real embeddings have hundreds or thousands of dimensions. This 2D picture shows the idea: similar meaning = nearby points.
Example: search by meaning, not keywords

Student asks: "How do I apply for placements?"

Sentence in the FAQSimilarity (illustrative)
What is the process for campus recruitment?0.82 (very close)
Steps to register with the training & placement cell0.78
Placement of furniture in the new hostel block0.21 (far)
The first sentence shares almost no words with the question, but it means the same thing. The last one shares the word "placement" but means something else. Embeddings get this right; keyword search does not.
Famous example With word embeddings you can do arithmetic on meaning: king − man + woman ≈ queen. The direction "man → woman" is roughly the same as "king → queen".

Where embeddings are used: semantic search, recommendations ("students who liked this course also liked..."), finding duplicate questions, and RAG, which we build in Session 4.

# Compare sentence meanings (run in Google Colab)
!pip install sentence-transformers
from sentence_transformers import SentenceTransformer, util

model = SentenceTransformer("all-MiniLM-L6-v2")
q = "How do I apply for placements?"
faqs = ["What is the process for campus recruitment?",
        "Placement of furniture in the new hostel block"]

scores = util.cos_sim(model.encode(q), model.encode(faqs))
for faq, s in zip(faqs, scores[0]):
    print(f"{s:.2f}  {faq}")

6. Context window

The context window is the maximum number of tokens a model can look at in one go. It includes everything: the instructions, the whole conversation so far, any documents you pasted, and the answer it's writing.

Analogy: an open-book exam with a small desk You can bring notes, but your desk only fits a certain number of pages. If you add more, the oldest pages fall off. The model can only "see" what's on the desk right now.

7. Temperature

Remember that the model gives a probability for every possible next token. Temperature controls how it picks from that list.

Try it: move the slider
TaskTemperatureWhy
Answer from the syllabus, extract data to JSON, write codeLow (0 – 0.3)You want the same correct answer every time.
General chat, explanationsMedium (0.5 – 0.8)Natural but still focused.
Names for the college tech fest, poems, brainstormingHigh (0.9 – 1.2)You want variety and surprise.

You may also see top-p. It's a related setting: "only pick from the smallest group of words that together cover p% of the probability". Usually you tune one of the two, not both.

8. Why LLMs hallucinate

A hallucination is when a model gives an answer that sounds confident and fluent but is false or made up.

Example

Prompt: "Summarise the 2019 research paper 'Quantum Biryani Optimization' by Dr. R. Sharma, IIT Bombay."

This paper doesn't exist. A model may still write a detailed summary, with methods and results, because that's what a summary of a paper looks like. Other common cases: fake citations, wrong dates, invented court cases, made-up library functions in code.

Why it happens

  1. It predicts plausible text, not true text. Its goal is "what words usually come next", not "what is a fact".
  2. It has no built-in fact database. What it learned is stored as patterns in its parameters, not as a list of facts it can check.
  3. Knowledge cutoff. It only knows what was in its training data up to a certain date. It doesn't know your college's latest timetable.
  4. It is rewarded for answering. Training often rewards a helpful-looking answer more than "I don't know", so it guesses.

How to reduce it

9. Open vs closed models, and how to choose

Closed modelsOpen-weight models
ExamplesGPT (OpenAI), Claude (Anthropic), Gemini (Google)Llama (Meta), Mistral, also Gemma (Google), Qwen (Alibaba), DeepSeek
How you use itThrough the company's app or API. You can't download the model.Download and run it yourself: on a laptop (e.g. with Ollama), your own server, or a host like Groq.
StrengthsUsually the most capable. No hardware needed. Quick to start.Full control. Data can stay on your machines. Can be customised (fine-tuned). No per-token fee if self-hosted.
WeaknessesPay per token. Your data goes to the provider. The provider can change or retire the model.Need GPUs and skills to run well. Big models are expensive to host. Licences vary.

"Open-weight" means the trained model is published. It doesn't always mean the training data and code are open too, and licences can restrict commercial use. Always read the licence.

How to choose: a simple checklist

  1. Quality: is it good enough at your task? Test with 10–20 real examples.
  2. Cost: how many requests per day × tokens per request × price per token?
  3. Speed: does the user need an answer in under a second (chat) or can it wait (overnight report)?
  4. Privacy: can this data leave your organisation?
  5. Language: does it handle Hindi, Kannada, Tamil, etc. well?
  6. Context length: do you need to fit long documents?
  7. Licence: are you allowed to use it commercially?
Scenarios: which would you pick?
24-hour hackathon prototypeA closed model through an API, or a free tier (Gemini free tier, Groq). Fastest to start; quality matters more than cost here.
Hospital summarising patient records; data must not leave the buildingAn open-weight model running on the hospital's own servers.
Classifying 10 lakh customer reviews a day as positive / negativeA small, cheap, fast model. A giant model is overkill for a simple task at high volume.
Customer support chatbot in KannadaTest several models on real Kannada questions before choosing. Language quality varies a lot.
Practical tip Start with the strongest model you can access to prove the idea works. Then try smaller, cheaper models and keep the smallest one that's still good enough.

10. Recap

  1. AI ⊃ ML ⊃ Deep Learning ⊃ LLMs. ML learns rules from examples.
  2. Transformers use attention to understand each word using the words around it.
  3. An LLM predicts the next token, again and again. Training stages turn it into an assistant.
  4. Tokens are the unit of input, cost and limits. Embeddings turn meaning into numbers.
  5. The context window is the model's short-term memory. LLMs don't remember between calls.
  6. Temperature: low for facts, high for creativity.
  7. LLMs hallucinate because they generate plausible text. Ground them with sources.
  8. Choose models by quality, cost, speed, privacy, language and licence, not by hype.

Quick quiz

1. An ML spam filter is given 10,000 labelled emails. What does it produce?A model: rules/patterns it learned to decide whether a new email is spam.
2. What does "attention" help a transformer do?Decide which other words in the input matter for understanding each word (e.g. "bank" near "money" vs near "river").
3. Roughly how many tokens is a 750-word English essay?About 1,000 tokens (1 token ≈ ¾ word).
4. Why does a chatbot "forget" the start of a very long conversation?The conversation exceeded the context window, so older messages were dropped or summarised.
5. You're extracting student names and marks into JSON. High or low temperature?Low (around 0). You want consistent, exact output.
6. Give two reasons LLMs hallucinate and one way to reduce it.They predict plausible text, not verified facts; they have a knowledge cutoff; training rewards answering. Reduce it by giving source documents (RAG), allowing "I don't know", lowering temperature, or checking citations.
7. A bank wants an internal assistant, and customer data can't leave its servers. Open or closed model?An open-weight model hosted on the bank's own infrastructure (or a closed model only if the provider offers a deployment that meets the bank's data rules).

Glossary

TermMeaning in one line
ModelThe trained program that makes predictions.
ParametersThe numbers inside a model that store what it learned.
Neural networkLayers of simple maths units that learn patterns from data.
TransformerThe neural network design behind LLMs; uses attention.
TokenA chunk of text the model reads and writes.
EmbeddingA list of numbers representing the meaning of text.
Context windowMax tokens the model can see at once (input + output).
TemperatureHow random or creative the model's word choice is.
HallucinationA confident but false answer.
Knowledge cutoffThe date after which the model has no training data.
Pre-training / fine-tuningLearning from huge general text / then from smaller, focused examples.
RLHFReinforcement Learning from Human Feedback: humans rank answers to improve the model.
Open-weight modelA model whose trained parameters you can download and run.

Learn more (free)

Instructor guide

Suggested plan for the 60 minutes. Press S in the slides to open speaker notes.

TimeSegmentWhat to do
0–10 minIntro + hookIntroduce yourself. Ask: who is worried about layoffs or AI taking jobs? Walk through why layoffs are happening, what to improve and what to learn. Keep it encouraging, not scary. Then: who used ChatGPT / Gemini / Claude this week? "How does it know?"
10–18 minAI → ML → DL → TransformersSpam filter, placement prediction, face unlock, "bank" and "it" examples.
18–22 minNext-word predictionGame: say "Virat Kohli is famous for playing..." and let the class finish it. Then "Once upon a..." The class is the LLM.
22–35 minTokens, embeddings, context, temperatureLive tokenizer demo (English vs Hindi). Temperature slider demo. Desk analogy for context.
35–45 minHallucinationLive: ask a chatbot about the fake "Quantum Biryani Optimization" paper, or about a fake event at your college. Discuss why.
45–55 minOpen vs closed, choosingRun the four scenarios as a class vote before revealing answers. If short on time, do two scenarios.
55–60 minRecap + quizAsk 3–4 quiz questions aloud. Preview Session 2: prompt engineering.
Before class Open the tokenizer site and a chatbot in browser tabs. Test the hallucination prompt beforehand (newer models may refuse; if so, show that as a good example of a model saying "I can't find that"). Have a backup screenshot in case the internet is slow.