GenAI & Agentic AI Workshop
Day 1 ยท Session 3 ยท 1 hour

Building with LLM APIs

So far we've used chat apps. Now we call the model from our own Python code, so we can build our own apps. By the end you'll have a working chatbot that remembers the conversation.

Open the slides Jump to the lab
What you will learn
  1. What is an API?
  2. Your API key, and keeping it secret
  3. Your first API call
  4. Messages and roles
  5. Chat memory: the model forgets, your code remembers
  6. Streaming
  7. Tokens and cost
  8. Errors and rate limits
  9. Other providers look the same
  10. Lab: chatbot with memory
  11. Recap, quiz and glossary

1. What is an API?

An API (Application Programming Interface) is a way for one program to ask another program to do something. You send a request, you get back a response.

Analogy: ordering food You (your code) don't go into the kitchen (the model). You give your order to the waiter (the API), and the waiter brings back your food (the answer). You don't need to know how the kitchen works, only how to order.
Chat app (ChatGPT, Gemini)API
Who uses itA person, typingYour program
What you can buildNothing; you just chatYour own apps: bots, tools, automations
MemoryBuilt inYou build it yourself (section 5)
CostFree or monthly planPay per token (free tiers exist)
Your Python code sends a request (API key, model, messages, settings) to the LLM API, which sends back a response (text, token usage, finish reason).

2. Your API key, and keeping it secret

An API key is a long secret string that proves the request is from you. Whoever has your key can use your account and your quota.

  1. Get a free Gemini key at Google AI Studio (no credit card), or a Groq key at console.groq.com.
  2. In Google Colab, click the ๐Ÿ”‘ Secrets icon, add a secret named GEMINI_API_KEY, paste the key and switch on Notebook access.
  3. Read it in code with userdata.get("GEMINI_API_KEY").
Never do this

api_key = "AIzaSy..." written in your code and pushed to GitHub. Bots scan public repos and find leaked keys within minutes.

Keep keys in Colab Secrets, or in a .env file that is listed in .gitignore. If a key leaks, delete it in the console and make a new one.

3. Your first API call

!pip install -q google-genai
from google import genai
from google.colab import userdata

client = genai.Client(api_key=userdata.get("GEMINI_API_KEY"))
MODEL = "gemini-2.5-flash"   # any current free model from Google AI Studio

resp = client.models.generate_content(
    model=MODEL,
    contents="Explain what a compiler does in 2 lines, for a first-year student.",
)
print(resp.text)

That's it: a client, a model name, and your text. Everything else is optional.

Adding settings

Use a config to add a system prompt (Session 2), temperature (Session 1) and a limit on answer length:

from google.genai import types

resp = client.models.generate_content(
    model=MODEL,
    contents="Suggest 3 names for our college coding club.",
    config=types.GenerateContentConfig(
        system_instruction="You are a creative assistant. Reply as a numbered list.",
        temperature=1.0,          # creative task, so higher
        max_output_tokens=200,    # stop after ~150 words
    ),
)
print(resp.text)
What's in the response?

4. Messages and roles

A chat is a list of messages. Each message has a role that says who wrote it:

RoleWhoExample
systemYou, the developer (hidden rules)"You are CampusBuddy. Answer only college questions."
userThe person chatting"When is the DBMS exam?"
model / assistantThe AI's earlier replies"DBMS is on 12 Dec at 10 AM."

Gemini calls the AI's role model and passes the system prompt in the config. OpenAI-style APIs (Groq, OpenAI) call it assistant and put the system prompt as the first message. Same idea, different names.

5. Chat memory: the model forgets, your code remembers

Remember from Session 1: LLMs have no memory between calls. Each request starts fresh. So how does ChatGPT remember your name? The app re-sends the whole conversation every time.

Turn 1 sends system and user message. Turn 2 sends system, the turn 1 user message, the model's reply and the new question, so the model knows the name. Without memory, turn 2 sends only the new question and the model doesn't know the name.

In code, memory is just a Python list that grows:

SYSTEM = "You are CampusBuddy, a friendly helpdesk bot for ABC Engineering College."
history = []   # the whole conversation lives here

def chat(user_text):
    history.append({"role": "user", "parts": [{"text": user_text}]})
    resp = client.models.generate_content(
        model=MODEL,
        contents=history,                       # send EVERYTHING so far
        config=types.GenerateContentConfig(system_instruction=SYSTEM),
    )
    history.append({"role": "model", "parts": [{"text": resp.text}]})
    return resp.text

print(chat("Hi, I'm Priya from CSE."))
print(chat("What's my name and branch?"))   # โ†’ Priya, CSE
Try it: break the memory Change contents=history to contents=user_text and ask "What's my name?" again. The bot forgets. That one line is the memory.

The catch: long chats get expensive

Every turn re-sends all earlier turns, so the input grows. Turn 20 might send 5,000 tokens just to answer "ok thanks". Two simple fixes:

The SDK also has a shortcut, client.chats.create(...), that keeps the history for you. We build it by hand first so you know what's happening inside.

6. Streaming

A long answer can take several seconds. Without streaming, the user stares at a blank screen and then everything appears at once. With streaming, words appear as the model writes them, like in ChatGPT.

Without streaming: a long wait then the full answer. With streaming: a short wait, then words appear piece by piece. Same total time and cost.
for chunk in client.models.generate_content_stream(
    model=MODEL,
    contents="Give 5 tips to prepare for campus placements.",
):
    print(chunk.text or "", end="", flush=True)

Same cost, same total time. It just feels much faster. Use it whenever a person is waiting for the answer.

7. Tokens and cost

APIs charge per token (Session 1), with separate prices for input (what you send) and output (what the model writes). Output usually costs several times more than input.

u = resp.usage_metadata
print("Input tokens: ", u.prompt_token_count)
print("Output tokens:", u.candidates_token_count)
print("Total:        ", u.total_token_count)
Example: what would a campus chatbot cost?

Prices change, so always check the provider's pricing page. Say a model costs $0.30 per million input tokens and $2.50 per million output tokens (illustrative).

CalculationTokens per dayCost per day
Input2,000 students ร— 10 questions ร— 500 tokens10 million$3.00
Output2,000 students ร— 10 questions ร— 200 tokens4 million$10.00
Total$13 a day (roughly โ‚น1,100)
Notice the 500 input tokens per question: that includes the system prompt and chat history, which are re-sent every turn.
PRICE_IN, PRICE_OUT = 0.30, 2.50   # $ per million tokens (check current prices)

def cost(resp):
    u = resp.usage_metadata
    return (u.prompt_token_count * PRICE_IN + (u.candidates_token_count or 0) * PRICE_OUT) / 1_000_000

print(f"This answer cost ${cost(resp):.6f}")

How to keep costs down

Some newer models also "think" before answering. Those thinking tokens are billed as output too, so a reasoning model can cost more for the same visible answer.

8. Errors and rate limits

Real apps must handle failures. The common ones:

ErrorWhat it meansWhat to do
400 Bad requestSomething wrong in your request (bad model name, bad format)Fix your code
401 / 403Key missing, wrong or not allowedCheck the key and its secret name
429 Too many requestsYou hit the rate limit (requests per minute) or daily quotaWait and retry; slow down
500 / 503Problem on the provider's sideWait and retry
import time
from google.genai import errors

def ask_with_retry(prompt, tries=3):
    for i in range(tries):
        try:
            return client.models.generate_content(model=MODEL, contents=prompt).text
        except errors.APIError as e:
            if e.code in (429, 500, 503) and i < tries - 1:
                wait = 5 * 2 ** i          # 5s, then 10s, then 20s
                print(f"Error {e.code}, retrying in {wait}s...")
                time.sleep(wait)
            else:
                raise

Waiting longer after each failure is called exponential backoff. Free tiers have low rate limits, so in a class of 60 students you may see 429s. That's normal; wait a minute and try again.

9. Other providers look the same

Every LLM API follows the same pattern: model + messages + settings โ†’ text + token usage. Once you know one, the others take minutes. Here is the same call on Groq (fast, free tier, open models like Llama), which uses the OpenAI-style format:

!pip install -q groq
from groq import Groq

groq = Groq(api_key=userdata.get("GROQ_API_KEY"))
resp = groq.chat.completions.create(
    model="llama-3.3-70b-versatile",   # check console.groq.com for current models
    messages=[
        {"role": "system", "content": "You are CampusBuddy."},
        {"role": "user", "content": "Explain what a compiler does in 2 lines."},
    ],
)
print(resp.choices[0].message.content)
print(resp.usage.prompt_tokens, resp.usage.completion_tokens)

This is handy as a backup: if one provider is down or rate-limited during the workshop, switch to the other.

10. Lab: chatbot with memory (25 min)

Build CampusBuddy, a chatbot that remembers the conversation, streams its answers and tells you what each turn cost.

Open the lab in Colab Project 2 details
  1. Make your first call and print the token usage.
  2. Write a cost() function.
  3. Stream an answer.
  4. Build chat() with a history list. Prove it remembers your name.
  5. Break the memory on purpose, then fix it.
  6. Run a chat loop: type questions, type quit to stop. Print tokens and cost after each turn.
Stretch goals Keep only the last 6 messages and watch the input tokens stop growing. Add retry for 429 errors. Switch the same bot to Groq.

11. Recap

  1. An API lets your code send a request to a model and get a response.
  2. Your API key is a password. Keep it in Secrets or .env, never in GitHub.
  3. A call needs a model and contents. Settings add a system prompt, temperature and max tokens.
  4. Messages have roles: system, user, model/assistant.
  5. Memory = your code re-sends the whole history every turn. Trim it to save cost.
  6. Streaming shows words as they're written. Same cost, feels faster.
  7. Cost = input tokens ร— input price + output tokens ร— output price.
  8. Handle 429 and 5xx errors with retry and backoff.

Quick quiz

1. What three things does the simplest API call need?A client with your API key, a model name, and the contents (your text).
2. Your friend pasted their API key into code and pushed it to a public GitHub repo. What should they do?Delete (revoke) the key in the provider's console immediately, create a new one, and store it in Secrets or a .env file listed in .gitignore.
3. How does a chatbot "remember" your name if the model has no memory?The app keeps the conversation in a list and sends the whole list with every request.
4. Why do long chats cost more per question?Every turn re-sends all the earlier messages as input tokens, so input grows each turn.
5. Does streaming make the answer cheaper?No. Same tokens, same cost, same total time. It just shows words sooner, so it feels faster.
6. You get error 429. What does it mean and what do you do?Too many requests: you hit the rate limit or quota. Wait and retry with backoff, and slow down.
7. 1,000 users ร— 5 questions ร— 400 input tokens a day, at $0.30 per million input tokens. Input cost per day?2,000,000 tokens ร— $0.30 / 1,000,000 = $0.60 a day.

Glossary

TermMeaning in one line
APIA way for your program to ask another program to do something.
API keyA secret string that proves a request is from you.
SDKA library (like google-genai) that makes calling an API easy.
Request / responseWhat you send / what you get back.
RoleWho wrote a message: system, user or model/assistant.
Chat historyThe list of earlier messages your code re-sends for memory.
StreamingReceiving the answer piece by piece as it's written.
Input / output tokensTokens you send / tokens the model writes. Both are billed.
Rate limitThe maximum number of requests allowed per minute or day.
Exponential backoffRetrying after waiting longer each time (5s, 10s, 20s...).

Learn more (free)

Instructor guide

Suggested plan for the 60 minutes. Press S in the slides to open speaker notes.

TimeSegmentWhat to do
0โ€“5 minHookAsk: how would you put ChatGPT inside your own app? Waiter analogy. Show the request/response diagram.
5โ€“12 minAPI key + first callEveryone gets a Gemini key and adds it to Colab Secrets. Live-code the first call. Warn about pushing keys to GitHub.
12โ€“22 minRoles + memoryShow the memory diagram. Live: chat() remembers "Priya", then break it by sending only the new message.
22โ€“30 minStreaming + costStream a long answer. Print usage. Do the 2,000-students cost example on the board.
30โ€“33 minErrorsShow the error table. Explain that 429s in class are normal.
33โ€“55 minLabStudents build the chatbot. Walk around; most issues are the secret name or Notebook access being off.
55โ€“60 minRecap + quizQuiz questions 3, 4 and 7. Preview Session 4: RAG, so the bot answers from college documents.
Before class Check the current free model name in Google AI Studio and update MODEL in the notebook if needed. Have a Groq key ready as a backup. With a large class, stagger the first run (row by row) to avoid rate limits.