GenAI & Agentic AI Workshop · Day 1 · Session 3

Building with LLM APIs

From chatting with AI to building with it

Press → to move · S for speaker notes · F for full screen

How do apps put AI inside them? 🤔

Food-delivery support bots, review summaries, your college's future helpdesk...

They don't open ChatGPT.
Their code calls an API.

API = the waiter 🍽️

You = your code

Place an order (request)

Kitchen = the model

Runs on Google's / Groq's servers

The waiter (API) takes the order and brings back the food (response). You don't need to know how the kitchen works.

One API call

Request with key, model, messages, settings; response with text, usage, finish reason

Your API key = a password 🔑

  1. Get a free key: aistudio.google.com/apikey
  2. Colab → 🔑 Secrets → GEMINI_API_KEY → Notebook access ON
  3. In code: userdata.get("GEMINI_API_KEY")

🚫 Never paste the key in code and push to GitHub. Bots find leaked keys in minutes.

Your first call

from google import genai
from google.colab import userdata

client = genai.Client(api_key=userdata.get("GEMINI_API_KEY"))
MODEL = "gemini-2.5-flash"

resp = client.models.generate_content(
    model=MODEL,
    contents="Explain what a compiler does in 2 lines, for a first-year student.",
)
print(resp.text)

Client + model + text. That's it.

Adding settings

resp = client.models.generate_content(
    model=MODEL,
    contents="Suggest 3 names for our college coding club.",
    config=types.GenerateContentConfig(
        system_instruction="You are a creative assistant. Reply as a numbered list.",
        temperature=1.0,          # Session 1
        max_output_tokens=200,    # limit length (and cost)
    ),
)

Response also has usage_metadata (tokens) and finish_reason (STOP or MAX_TOKENS).

Messages have roles

RoleWhoExample
systemDeveloper (hidden)"You are CampusBuddy..."
userThe person"When is the DBMS exam?"
model / assistantThe AI's earlier replies"12 Dec at 10 AM."

Gemini says model; OpenAI-style APIs (Groq) say assistant.

Memory: the model forgets 🐟

Re-sending history gives memory; without history the model forgets

Memory is just a list

history = []

def chat(user_text):
    history.append({"role": "user", "parts": [{"text": user_text}]})
    resp = client.models.generate_content(
        model=MODEL, contents=history,        # send EVERYTHING
        config=types.GenerateContentConfig(system_instruction=SYSTEM))
    history.append({"role": "model", "parts": [{"text": resp.text}]})
    return resp.text

chat("Hi, I'm Priya from CSE.")
chat("What's my name and branch?")   # → Priya, CSE ✅

Change contents=history → contents=user_text and it forgets ❌

Streaming ⚡

Streaming shows words as they are generated
for chunk in client.models.generate_content_stream(model=MODEL, contents=prompt):
    print(chunk.text or "", end="", flush=True)

What does it cost? 💰

Illustrative prices: $0.30 / million input tokens, $2.50 / million output tokens

2,000 students × 10 questions a dayTokensCost
Input× 500 tokens (prompt + history)10 M$3.00
Output× 200 tokens4 M$10.00
Total per day$13 ≈ ₹1,100

Save: smaller model · short system prompt · trim history · max_output_tokens · cache common answers

When things go wrong

ErrorMeaningFix
400Bad requestFix your code
401 / 403Bad or missing keyCheck the secret
429Rate limit hitWait, retry with backoff
500 / 503Provider problemWait, retry

60 students on free keys = some 429s. Normal! Wait a minute and retry.

Every API looks the same

model + messages + settings
→ text + token usage

Gemini

client.models.generate_content(...)

Groq / OpenAI

client.chat.completions.create(...)

Keep a Groq key as a backup if one provider is slow or rate-limited.

Lab: CampusBuddy with memory (25 min) 🧪

  1. First call + print token usage
  2. Write cost()
  3. Stream an answer
  4. Build chat() with history. Prove it remembers you
  5. Break the memory, then fix it
  6. Chat loop with tokens + cost per turn

Stretch: keep only the last 6 messages · retry on 429 · switch to Groq

Recap

  1. API: your code → request → response
  2. API key = password. Never in GitHub
  3. Roles: system · user · model
  4. Memory = re-send the history (trim it!)
  5. Streaming = same cost, feels faster
  6. Cost = input × price + output × price
  7. 429 → wait and retry

Quick quiz 🧠

  1. How does a bot remember your name? Code re-sends the history
  2. Why do long chats cost more? More input tokens every turn
  3. Is streaming cheaper? No, just feels faster
  4. Error 429? Rate limit: wait and retry

Up next · Session 4

RAG

Make CampusBuddy answer from your college's own documents, with sources

Questions? 🙋