GenAI & Agentic AI Workshop · Day 1 · Session 4
Retrieval-Augmented Generation: make the AI answer from your documents
Press → to move · S for speaker notes · F for full screen
"What is the revaluation fee at ABC Engineering College, and how many days do I have to apply?"
It has never seen our rules.
So it guesses, or gives up.
Fine for 2 pages.
500 pages? Too many tokens, slow, costly.
Retrain the model.
Costly, redo on every rule change, still hallucinates.
Search the documents.
Send only the 2–3 pieces that matter.
Find the pieces of your documents that match the question
Add those pieces to the prompt
The LLM writes the answer from them, with sources
An open-book exam 📖
from pypdf import PdfReader
reader = PdfReader("syllabus.pdf")
pages = [page.extract_text() for page in reader.pages]
⚠️ Garbage in, garbage out: jumbled tables, repeated headers, and scanned PDFs give no text (need OCR).
from sentence_transformers import SentenceTransformer
embedder = SentenceTransformer("all-MiniLM-L6-v2") # free, runs in Colab
vectors = embedder.encode(chunks, normalize_embeddings=True)
🚨 Same embedding model for chunks and questions. Different models = different maps.
| Option | Good for |
|---|---|
| Chroma | Learning, prototypes (today) |
| FAISS | Big collections on your own machine |
| pgvector | Apps already on PostgreSQL |
| Managed / cloud | Production: Pinecone, Vertex AI, Azure AI Search, OpenSearch |
Stores vector + text + source. Finds the closest vectors fast.
| Question | Top chunk |
|---|---|
| Is the library open on Sunday? | [Library] "...closed on Sundays, except exam months..." |
| More drives after a 7 LPA offer? | [Placements] "...one-student-one-job policy..." |
| Who won the IPL in 2020? | Nothing relevant: all scores low ❌ |
Use a minimum score: if nothing is close, don't send junk to the model.
SYSTEM: Answer using ONLY the context below.
- After each fact, cite the source in [brackets].
- If it's not in the context, say "I couldn't find that in the college documents."
Context:
[Exams] ...hall ticket only with at least 75% attendance...
[Attendance] ...65% to 74%: condonation with medical certificate, fee ₹1,000...
Question: My attendance in DBMS is 70%. Can I still write the exam?
You need 75% for a hall ticket [Exams]. With 70%, apply for condonation with a medical certificate and ₹1,000 [Attendance].
| Symptom | Fix |
|---|---|
| Wrong or vague answer | Print the retrieved chunks first. Fix chunking / k |
| "Not found" but it's there | Check extracted text; lower the threshold |
| Answers off-topic questions | "ONLY the context" + minimum score |
| Old information | Re-index when documents change |
| Can't find "CS301" | Hybrid search: keywords + vectors |
Most RAG bugs are retrieval bugs.
Stretch: upload your college's PDF · tune chunk size, k, min score · add memory
Up next · Session 5
Hallucination, bias, privacy, prompt injection and guardrails. Then Day 2: agents!
Questions? 🙋