Week 14: Generative AI and LLMs — Using the Claude API with Python

Month 4 | Main goal: The student understands what an LLM is, uses the Claude API securely (API key in .env), builds a Swahili chatbot in the terminal, and then puts AI inside a FastAPI endpoint.

Teacher Preparation (read before the week starts)

• IMPORTANT — do this early, before Session 1:
  1. Open an account on the Claude Console: https://platform.claude.com (formerly console.anthropic.com).
  2. Add a small balance (credits): e.g. USD 5 is enough for a full week of practice with the low-cost model.
  3. Set a spend limit in Console → Settings → Limits, so the student doesn't spend a lot of money by accident.
  4. Create an API key (Console → API Keys → Create Key). Give it a name like mwanafunzi-wiki14 (student-week14). Copy it immediately: it is not shown again.
  5. Give the student that key privately (not in a recorded Google Meet chat, not on screen share). At the end of the course you can delete it (revoke).
• Read:
  • Python SDK quickstart: https://platform.claude.com/docs/en/api/client-sdks
  • Messages API: https://platform.claude.com/docs/en/api/messages
  • Streaming: https://platform.claude.com/docs/en/build-with-claude/streaming
  • Tool use: https://platform.claude.com/docs/en/agents-and-tools/tool-use/overview
  • Pricing: https://platform.claude.com/docs/en/about-claude/pricing

• Install:

  powershell

  Copy

  mkdir wiki14_ai
  cd wiki14_ai
  python -m venv venv
  .\venv\Scripts\Activate.ps1
  pip install anthropic python-dotenv fastapi "uvicorn[standard]" pytest httpx
  

• Models we will use: | Model ID | Use | Price (USD per 1 million tokens: input / output) | |----------|----------|------| | claude-haiku-5-5 | Classroom practice: fast and very cheap | $0.10 / $0.50 | | claude-sonnet-5-5 | Highest quality (hard tasks) | $2 / $10 |

  Advice: in class use claude-haiku-5-5 nearly all the time. We will show claude-sonnet-5-5 only once, for comparison. Prices can change: check the pricing page before class.

• Don't show your API key on screen share! Before sharing your screen, close the .env file. If you show it by accident: go to the Console, delete it (revoke), and create a new one.

Week Goals (by the end of the week the student will be able to...)

1. Explain in simple language what an LLM (Large Language Model) is, and what a token is.
2. Write a good prompt (clear, with context, with an example answer).
3. Store an API key in .env, read it with python-dotenv, and make sure .env doesn't get into Git.
4. Call the Claude API with the anthropic SDK (client.messages.create) and read the response, stop_reason and usage.
5. Calculate the cost of a request from its tokens.
6. Use a system prompt and conversation history (multi-turn).
7. Use streaming so the answer appears word by word.
8. Understand the concepts of tool use and structured JSON output.
9. Build a Swahili chatbot in the terminal, and a FastAPI endpoint POST /uliza (ask).
10. Know about a free alternative (Ollama: models that run on your own computer).

Session 1 — What is an LLM? API key and the first request

Session flow

1. Review of the Week 13 homework (exam scores)
2. What is an LLM? Tokens, prompts, how it differs from the ML of Week 13
3. API keys, .env, .gitignore: security first
4. Live coding: the first request, reading the response, usage and cost
5. Errors and how to catch them
6. In-class exercise
7. Q&A + homework

Explanation for the teacher (read this first)

Week 13 vs Week 14. Last week we trained models ourselves (small, for one task, e.g. the price of a plot). This week we use a very large model already trained by a company (Anthropic) on an enormous amount of the world's text. We don't train it: we call it over the internet (API) and pay for usage.

An LLM (Large Language Model) is an ML model trained to predict the next piece of text. By doing this at a very large scale, it has learned language, many facts, and even step-by-step reasoning. "Generative AI" = AI that generates new content: text, code, images.

Token. An LLM doesn't read letters or whole words: it reads pieces called tokens. In English, one token ≈ three quarters of a word. In Swahili, one word is often split into more pieces (e.g. "tunawapenda" (we love them) → several pieces), so Swahili text often has more tokens than the same text in English. This matters because we pay by tokens: both incoming (input) and outgoing (output). Output is more expensive.

Prompt = the instructions/question we give the model. A good prompt: (1) is clear and specific, (2) has context (who is asking, why), (3) describes the shape of the answer (e.g. "a list of 3 items", "2 sentences"), (4) may include an example.

An API key is like a password + bank card combined: anyone with your key can use your balance. Therefore:

• It is not written inside the code (api_key="sk-ant-...": NO).
• It goes in a .env file, which is added to .gitignore before the first commit.
• python-dotenv reads .env and puts the values into "environment variables". The anthropic SDK automatically looks for a variable called ANTHROPIC_API_KEY.

Structure of a request:

client.messages.create(
    model="claude-haiku-5-5",   # model ipi
    max_tokens=1024,            # kikomo cha juu cha tokens za jibu
    messages=[{"role": "user", "content": "..."}],  # mazungumzo
)

The response is an object with:

• response.content: a list of "blocks". A text block has type == "text" and .text. Modern models may also return a "thinking" block (internal reasoning) before the text, so we filter for "text" blocks rather than assuming content[0] is the text.
• response.stop_reason: why it stopped: "end_turn" (finished normally), "max_tokens" (was cut off: increase max_tokens), "tool_use" (it wants to use a tool: Session 3), "refusal" (it declined to answer for safety reasons).
• response.usage.input_tokens, response.usage.output_tokens: for calculating the cost.

Effort (the amount of "thinking"). These models think a little before answering, and thinking tokens count as output (they are paid for). For simple conversations we can reduce it with output_config={"effort": "low"}: a faster and cheaper answer. For hard tasks, leave the default.

Say this to the student

"Last week we taught our little 'broker' to predict plot prices. This week we are using a 'professor' who has read the whole library of the world. We don't teach him; we just ask him and pay for every word he writes."

"An LLM is like a finish-the-sentence game. If I say 'Mama is cooking ugali and...' you say 'vegetables' or 'fish'. An LLM has played that game billions of times, and has become very skilled."

"A token is like pieces of sugarcane: the model doesn't eat the whole cane, it cuts it into pieces. We pay for each piece."

"An API key is like your M-Pesa PIN. You wouldn't write your PIN on a public notice board. Public GitHub is a notice board! There are robots searching for leaked keys every minute."

Type this code live (share screen)

Before sharing your screen: prepare your .env in advance, and don't open it while sharing.

Step 1: .gitignore FIRST

git init

Create a .gitignore file:

venv/
.env
__pycache__/

Step 2: .env and .env.example The .env file (not to be shared):

ANTHROPIC_API_KEY=sk-ant-weka-key-yako-hapa

The .env.example file (this one can be committed; it shows others the required variables without the secrets):

ANTHROPIC_API_KEY=

Verify:

git status

.env must not appear in the list. If it does, .gitignore has a mistake.

Step 3: The first request — kwanza.py (first)

# kwanza.py
from dotenv import load_dotenv
import anthropic

load_dotenv()                     # inasoma .env -> ANTHROPIC_API_KEY
client = anthropic.Anthropic()    # inachukua key yenyewe kutoka mazingira

response = client.messages.create(
    model="claude-haiku-5-5",
    max_tokens=1024,
    messages=[
        {"role": "user", "content": "Habari! Nieleze Python ni nini kwa sentensi mbili, kwa Kiswahili."}
    ],
)

# response.content ni list ya blocks; tunachukua zile za maandishi tu
jibu = "".join(block.text for block in response.content if block.type == "text")
print(jibu)
print("-" * 40)
print("stop_reason:", response.stop_reason)
print("Tokens - input:", response.usage.input_tokens, "| output:", response.usage.output_tokens)

python kwanza.py

Output (the text will differ each time; that's normal):

Python ni lugha ya programu iliyo rahisi kusoma na kuandika, inayotumika kutengeneza
tovuti, uchambuzi wa data na akili bandia. ...
----------------------------------------
stop_reason: end_turn
Tokens - input: 28 | output: 74

Say: "Run it again: the answer will change a little. An LLM isn't like an ordinary function that returns the same thing every time."

Step 4: A helper function + cost — msaidizi.py (helper)

# msaidizi.py
from dotenv import load_dotenv
import anthropic

load_dotenv()
client = anthropic.Anthropic()

# Bei kwa USD kwa tokens MILIONI 1 (angalia ukurasa wa pricing, zinaweza kubadilika)
BEI = {
    "claude-haiku-5-5": {"input": 0.10, "output": 0.50},
    "claude-sonnet-5-5": {"input": 2.00, "output": 10.00},
}
USD_KWA_TSH = 2600   # takriban — badilisha kulingana na kiwango cha siku


def maandishi(response) -> str:
    """Rudisha maandishi yote kutoka kwenye jibu."""
    return "".join(b.text for b in response.content if b.type == "text")


def gharama(response, model: str) -> float:
    """Rudisha gharama ya ombi kwa USD."""
    b = BEI[model]
    return (response.usage.input_tokens * b["input"]
            + response.usage.output_tokens * b["output"]) / 1_000_000


def uliza(swali: str, model: str = "claude-haiku-5-5") -> str:
    response = client.messages.create(
        model=model,
        max_tokens=1024,
        messages=[{"role": "user", "content": swali}],
    )
    usd = gharama(response, model)
    print(f"[{model}] tokens in={response.usage.input_tokens} out={response.usage.output_tokens} "
          f"| gharama ≈ ${usd:.6f} (≈ TSh {usd * USD_KWA_TSH:.2f})")
    return maandishi(response)


if __name__ == "__main__":
    print(uliza("Nipe methali tatu za Kiswahili kuhusu bidii, na maana zake kwa ufupi."))

Output (approximately):

[claude-haiku-5-5] tokens in=30 out=180 | gharama ≈ $0.000093 (≈ TSh 0.24)
1. Haba na haba hujaza kibaba — ...

Say: "Less than one shilling! But with 10,000 users a day, the cost becomes noticeable. That's why we measure it."

Step 5: Counting tokens before sending (Swahili vs English)

from msaidizi import client

for sentensi in ["Tunawapenda sana wanafunzi wetu wote.",
                 "We love all our students very much."]:
    hesabu = client.messages.count_tokens(
        model="claude-haiku-5-5",
        messages=[{"role": "user", "content": sentensi}],
    )
    print(f"{hesabu.input_tokens:3d} tokens  <-  {sentensi}")

The output will show that Swahili has slightly more tokens for the same meaning. (The exact number depends on the model.) count_tokens is free.

Step 6: Reducing "thinking" for simple questions

response = client.messages.create(
    model="claude-haiku-5-5",
    max_tokens=1024,
    output_config={"effort": "low"},   # jibu la haraka na nafuu
    messages=[{"role": "user", "content": "2 + 2 ni ngapi? Jibu kwa namba tu."}],
)
print(maandishi(response), response.usage.output_tokens)

Step 7: Catching errors

import anthropic
from msaidizi import client, maandishi

try:
    r = client.messages.create(
        model="claude-haiku-5-5",
        max_tokens=200,
        messages=[{"role": "user", "content": "Sema habari"}],
    )
    print(maandishi(r))
except anthropic.AuthenticationError:
    print("API key si sahihi au haipo. Angalia faili .env")
except anthropic.RateLimitError:
    print("Maombi mengi mno kwa muda mfupi. Subiri kidogo ujaribu tena.")
except anthropic.APIConnectionError:
    print("Hakuna mtandao au server haipatikani.")
except anthropic.APIStatusError as e:
    print(f"Hitilafu ya API ({e.status_code}): {e.message}")

Show the key error on purpose: change .env to a fake key (without sharing .env!), run → "API key si sahihi..." (API key is not valid). Restore the correct key.
The SDK automatically retries 2 times for temporary errors (429, 5xx).

Questions to ask the student

1. Question: What is the big difference between the Week 13 model and an LLM? Answer: In Week 13 we trained a small model ourselves for one task; an LLM is already trained on very large data, we call it through an API and it can do many language tasks.

2. Question: Why shouldn't we write the API key inside kwanza.py? Answer: If we commit/share the code, the key leaks and someone can use our balance. We put it in .env, which is in .gitignore.

3. Question: What does stop_reason == "max_tokens" mean and what should we do? Answer: The answer was cut off before finishing; increase max_tokens or ask for a shorter answer.

4. Question: Why shouldn't we use response.content[0].text directly? Answer: content is a list that may have other kinds of blocks (e.g. thinking) at the start; we filter on type == "text".

5. Question: Which is more expensive: input tokens or output tokens? Answer: Output (5 times more for these models).

Common mistakes

In-class exercise

Exercise 1.1: Write a script tafsiri.py (translate) that asks Claude to translate 3 English sentences into Swahili. Print the translation and the cost.

Exercise 1.2: Ask the same question with max_tokens=20. Print stop_reason. Then with max_tokens=1024. Explain the difference.

Exercise 1.3: Write two prompts asking Claude to explain "variable" in Python: (a) a short unclear prompt: "variable?", (b) a good prompt with context and structure. Compare the answers.

# tafsiri.py
from msaidizi import client, maandishi, gharama

MODEL = "claude-haiku-5-5"

# 1.1
sentensi = """1. The market opens at seven in the morning.
2. Please send me the report by Friday.
3. Learning to code takes patience."""
r = client.messages.create(
    model=MODEL,
    max_tokens=500,
    messages=[{"role": "user", "content":
        f"Tafsiri sentensi hizi kwenda Kiswahili sanifu. Rudisha tafsiri tu, zikiwa na namba zile zile:\n{sentensi}"}],
)
print(maandishi(r))
print(f"Gharama: ${gharama(r, MODEL):.6f}")

# 1.2
for kikomo in [20, 1024]:
    r = client.messages.create(
        model=MODEL,
        max_tokens=kikomo,
        messages=[{"role": "user", "content": "Eleza historia ya Tanzania kwa aya mbili."}],
    )
    print(f"max_tokens={kikomo}: stop_reason={r.stop_reason}")
# 20 -> "max_tokens" (jibu limekatwa, au hata halina maandishi kama tokens ziliisha wakati wa kufikiri)
# 1024 -> "end_turn" (limekamilika)

# 1.3
mbaya = "variable?"
nzuri = ("Mimi ni mwanafunzi ninayeanza kujifunza Python. Nieleze 'variable' ni nini "
         "kwa Kiswahili rahisi, kwa sentensi 3, kisha nipe mfano mmoja mfupi wa code.")
for p in [mbaya, nzuri]:
    r = client.messages.create(model=MODEL, max_tokens=600,
                               messages=[{"role": "user", "content": p}])
    print(f"=== PROMPT: {p}\n{maandishi(r)}\n")
# Prompt nzuri inatoa jibu la Kiswahili, fupi, lenye mfano — kwa sababu tuliomba hivyo.

Session 2 — System prompt, conversation history and Streaming

Session flow

1. Review of the homework (Session 1)
2. System prompt: "giving the model a job/identity"
3. Multi-turn: the model doesn't remember; we send the history
4. A Swahili chatbot in the terminal
5. Streaming: the answer word by word
6. In-class exercise
7. Q&A + homework

Explanation for the teacher (read this first)

System prompt (system="...") is the general set of instructions that applies to the whole conversation: identity ("You are the assistant for Mama Neema's shop"), language ("answer in Swahili"), style ("short, friendly"), limits ("don't answer questions unrelated to the shop"). It is different from a user message: it is like a manager's instructions to an employee before the shop opens.

The model has NO memory. Each API request stands alone. For the model to "remember" a conversation, we send the whole history every time:

messages = [
    {"role": "user", "content": "Jina langu ni Juma."},
    {"role": "assistant", "content": "Karibu Juma!"},
    {"role": "user", "content": "Jina langu ni nani?"},   # sasa itajibu "Juma"
]

The rule: messages alternate user → assistant → user ..., and the first is user. The cost consequence: every turn we send the whole history again, so input tokens grow with every message. Long conversations = more expensive. That's why chatbots usually have a "start over" (reset) command or trim the history.

Saving the assistant's response in the history: The safest way is to put the whole response.content (the list of blocks) as the assistant's content. The SDK accepts it, and it keeps all the blocks (including thinking ones) as the API recommends.

Streaming. Without streaming, the user waits several seconds seeing nothing, and then the whole answer appears. With streaming, the words appear one by one as if someone were typing: a much better experience, and long answers don't get stuck on a "timeout". SDK:

with client.messages.stream(model=..., max_tokens=..., messages=...) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)
    final = stream.get_final_message()   # jibu kamili mwishoni (pamoja na usage)

flush=True forces the text to appear immediately in the terminal.

Say this to the student

"A system prompt is like the uniform and instructions you give a new restaurant waiter: 'Your name is Neema, greet customers in Swahili, don't talk about politics.' Then customers come with their questions."

"The model is like a person with total forgetfulness: every time you call him he has forgotten everything. So every time you give him the notebook of the whole earlier conversation to read first. If the notebook gets long, he takes longer to read it, and you pay more."

"Streaming is like watching a football match live instead of waiting for the final score."

Type this code live (share screen)

Step 1: System prompt — system_demo.py

from msaidizi import client, maandishi

MODEL = "claude-haiku-5-5"
swali = "Nina shilingi elfu kumi. Nifanye nini nazo?"

for system in [
    "Wewe ni mshauri wa fedha mwenye tahadhari. Jibu kwa Kiswahili, kwa sentensi 3.",
    "Wewe ni mpishi maarufu wa Zanzibar. Kila jibu lihusishe chakula. Jibu kwa Kiswahili, sentensi 3.",
]:
    r = client.messages.create(
        model=MODEL,
        max_tokens=400,
        system=system,
        messages=[{"role": "user", "content": swali}],
    )
    print(f"SYSTEM: {system}\n-> {maandishi(r)}\n")

Say: "Same question, completely different answers, just because of the system prompt."

Step 2: The model doesn't remember — kumbukumbu.py (memory)

from msaidizi import client, maandishi

MODEL = "claude-haiku-5-5"

# Ombi 1
r1 = client.messages.create(model=MODEL, max_tokens=200,
    messages=[{"role": "user", "content": "Jina langu ni Juma na ninaishi Arusha."}])
print("Bot:", maandishi(r1))

# Ombi 2 BILA historia
r2 = client.messages.create(model=MODEL, max_tokens=200,
    messages=[{"role": "user", "content": "Ninaishi wapi?"}])
print("Bila historia:", maandishi(r2))   # haijui!

# Ombi 2 PAMOJA na historia
r3 = client.messages.create(model=MODEL, max_tokens=200, messages=[
    {"role": "user", "content": "Jina langu ni Juma na ninaishi Arusha."},
    {"role": "assistant", "content": r1.content},
    {"role": "user", "content": "Ninaishi wapi?"},
])
print("Na historia:", maandishi(r3))     # "Arusha"

Step 3: A Swahili chatbot (CLI) with streaming — chatbot.py

# chatbot.py
from dotenv import load_dotenv
import anthropic

load_dotenv()
client = anthropic.Anthropic()

MODEL = "claude-haiku-5-5"
SYSTEM = (
    "Wewe ni 'Rafiki', msaidizi wa kirafiki anayezungumza Kiswahili sanifu cha Tanzania. "
    "Jibu kwa ufupi (sentensi 1-4) isipokuwa mtumiaji aombe maelezo marefu. "
    "Kama hujui jibu, sema hujui badala ya kubuni."
)


def main():
    historia = []
    jumla_in = jumla_out = 0
    print("Rafiki: Habari! Mimi ni Rafiki. Andika 'toka' kumaliza, au 'upya' kuanza upya.\n")

    while True:
        ujumbe = input("Wewe: ").strip()
        if not ujumbe:
            continue
        if ujumbe.lower() in {"toka", "exit", "quit"}:
            print(f"Rafiki: Kwaheri! (tokens jumla: in={jumla_in}, out={jumla_out})")
            break
        if ujumbe.lower() == "upya":
            historia = []
            print("Rafiki: Sawa, tumeanza upya.\n")
            continue

        historia.append({"role": "user", "content": ujumbe})
        try:
            print("Rafiki: ", end="", flush=True)
            with client.messages.stream(
                model=MODEL,
                max_tokens=2048,
                system=SYSTEM,
                output_config={"effort": "low"},
                messages=historia,
            ) as stream:
                for text in stream.text_stream:
                    print(text, end="", flush=True)
                final = stream.get_final_message()
            print("\n")
        except anthropic.APIError as e:
            print(f"\n[Hitilafu: {e}]\n")
            historia.pop()          # ondoa ujumbe ambao haukujibiwa
            continue

        historia.append({"role": "assistant", "content": final.content})
        jumla_in += final.usage.input_tokens
        jumla_out += final.usage.output_tokens


if __name__ == "__main__":
    main()

python chatbot.py

Example conversation:

Rafiki: Habari! Mimi ni Rafiki. Andika 'toka' kumaliza, au 'upya' kuanza upya.

Wewe: Jina langu ni Asha
Rafiki: Karibu sana Asha! Nikusaidie nini leo?

Wewe: Nipe wazo la biashara ndogo kwa mtaji wa laki moja
Rafiki: Unaweza kuanza biashara ya kuuza juisi ya matunda ...

Wewe: Jina langu ni nani?
Rafiki: Jina lako ni Asha.

Wewe: toka
Rafiki: Kwaheri! (tokens jumla: in=412, out=168)

Show: "Look at in=: it grows every turn because we send the whole history."

Questions to ask the student

1. Question: The difference between system and a user message? Answer: System is the general instructions on behaviour/identity for the whole conversation; user is the user's message for each turn.
2. Question: Why do input tokens grow every turn? Answer: We send the whole history with each request, because the model doesn't remember.
3. Question: Two benefits of streaming? Answer: The user sees the answer immediately (better experience); long answers don't get stuck on a timeout.
4. Question: What does the upya (restart) command save? Answer: It clears the history, so it reduces tokens (cost) and starts a new topic.
5. Question: Why historia.pop() in the except? Answer: So the history (historia) isn't left with a user message that has no answer (two user messages in a row).

Common mistakes

In-class exercise

Exercise 2.1: Change the chatbot into a "Maths Teacher" that answers step by step, in Swahili, and politely declines non-maths questions.

Exercise 2.2: Add a command gharama (cost) that prints the cost of the conversation so far (USD and TSh).

Exercise 2.3: Set a limit: the history must not exceed 10 messages (5 turns). If it does, delete the oldest turn (the first 2 messages).

# chatbot_mwalimu.py
from dotenv import load_dotenv
import anthropic

load_dotenv()
client = anthropic.Anthropic()

MODEL = "claude-haiku-5-5"
BEI_IN, BEI_OUT = 0.10, 0.50       # USD kwa tokens milioni 1
USD_KWA_TSH = 2600
MAX_UJUMBE = 10

# 2.1
SYSTEM = (
    "Wewe ni Mwalimu wa Hisabati mvumilivu. Jibu kwa Kiswahili, ukionyesha hatua kwa hatua "
    "(Hatua 1, Hatua 2, ...). Kama swali halihusu hisabati, kataa kwa adabu na mwalike "
    "mwanafunzi aulize swali la hisabati."
)


def main():
    historia = []
    jumla_in = jumla_out = 0
    while True:
        ujumbe = input("Wewe: ").strip()
        if not ujumbe:
            continue
        if ujumbe.lower() == "toka":
            break
        # 2.2
        if ujumbe.lower() == "gharama":
            usd = (jumla_in * BEI_IN + jumla_out * BEI_OUT) / 1_000_000
            print(f"Gharama hadi sasa: ${usd:.6f} ≈ TSh {usd * USD_KWA_TSH:.2f}\n")
            continue

        historia.append({"role": "user", "content": ujumbe})
        print("Mwalimu: ", end="", flush=True)
        with client.messages.stream(model=MODEL, max_tokens=2048, system=SYSTEM,
                                    messages=historia) as stream:
            for text in stream.text_stream:
                print(text, end="", flush=True)
            final = stream.get_final_message()
        print("\n")
        historia.append({"role": "assistant", "content": final.content})
        jumla_in += final.usage.input_tokens
        jumla_out += final.usage.output_tokens

        # 2.3 — futa zamu ya zamani zaidi (user + assistant) historia ikizidi
        while len(historia) > MAX_UJUMBE:
            historia = historia[2:]


if __name__ == "__main__":
    main()

Session 3 — Tool use, JSON output and an AI endpoint in FastAPI

Session flow

1. Review of the homework (Session 2)
2. Structured JSON output: getting data, not just words
3. Tool use: the model "asks" to use our functions
4. FastAPI: POST /uliza (ask), POST /uliza-stream, POST /chambua-maoni (analyse feedback)
5. Testing the endpoint without spending money (mock/monkeypatch)
6. Cost, security, and a free alternative (Ollama)
7. In-class exercise
8. Q&A + the week's homework

Explanation for the teacher (read this first)

Structured output (JSON). Often our program doesn't want a paragraph of words; it wants data: {"hisia": "chanya", "alama": 4} (hisia = sentiment, chanya = positive, alama = score). We give the API a JSON schema (a description of the structure) through output_config={"format": {"type": "json_schema", "schema": {...}}}, and the response is guaranteed to be JSON matching the schema. Then json.loads(...). The schema must have "additionalProperties": False and "required".

Tool use (function calling). The model can't know today's price of a product in our shop, nor write to our database. But we can tell it: "You have a tool called pata_bei (get price); if you need a price, ask me." The cycle:

1. We send the question + the tool descriptions (name, description, input_schema).
2. If the model decides it needs a tool, it returns stop_reason == "tool_use" and a tool_use block (name + input, e.g. {"bidhaa": "sukari"}, product: sugar).
3. We run our Python function and get the result.
4. We return the result as a tool_result block (with the same tool_use_id) in a user message.
5. The model uses the result to write the final answer.

   The model does not run our code: it only asks; we are the ones who decide and run it. This is important for security.

AI inside FastAPI. As in Week 13: we create the client once, the endpoint receives a Pydantic model, calls Claude, and returns JSON. For streaming we use FastAPI's StreamingResponse and a generator that emits (yield) pieces of text.

• Set limits: a max_length for the question (to stop someone sending huge text = cost), a max_tokens for the answer.
• Convert Anthropic errors into HTTPExceptions with appropriate statuses.
• Never return the API key in a response or logs.

Testing without cost: In pytest we don't want to call the real API (money + the answer changes). We use monkeypatch to replace main.client with a "fake client" that returns a prepared response.

Cost and free alternatives:

• Choose a cheap model (Haiku) for simple tasks; Sonnet only for hard tasks.
• Reduce max_tokens, the history, and the length of the system prompt; effort: "low" for simple questions.
• Set a spend limit in the Console.
• Ollama (https://ollama.com) runs open models (e.g. Llama, Gemma, Qwen) on your computer: free, without internet, but they need good RAM/GPU, are slower, and the quality (especially in Swahili) is usually lower than Claude.

Say this to the student

"Structured output is like giving someone a form to fill in instead of telling them 'write me a letter'. The form gives you organised data that is easy for a computer to read."

"Tool use is like a bank teller who tells you 'let me check your balance in the system'. She doesn't know your balance by heart: she uses a tool (the bank system). Claude is the teller; our functions are the system."

"When you put AI on an API, anyone in the world can use it, and every request costs you money. So we set up gates (limits), like a guard at the gate."

Type this code live (share screen)

Step 1: Structured JSON — json_demo.py

import json
from msaidizi import client, maandishi

SCHEMA = {
    "type": "object",
    "properties": {
        "hisia": {"type": "string", "enum": ["chanya", "hasi", "wastani"]},
        "alama": {"type": "integer", "description": "1 (mbaya sana) hadi 5 (nzuri sana)"},
        "muhtasari": {"type": "string"},
    },
    "required": ["hisia", "alama", "muhtasari"],
    "additionalProperties": False,
}

maoni = "Chakula kilikuwa kitamu sana ila huduma ilichelewa karibu saa nzima."

r = client.messages.create(
    model="claude-haiku-5-5",
    max_tokens=1024,
    system="Unachambua maoni ya wateja wa mgahawa. Muhtasari uwe kwa Kiswahili, sentensi moja.",
    messages=[{"role": "user", "content": maoni}],
    output_config={"format": {"type": "json_schema", "schema": SCHEMA}},
)
data = json.loads(maandishi(r))
print(data)
print("Hisia:", data["hisia"], "| Alama:", data["alama"])

Output (approximately):

{'hisia': 'wastani', 'alama': 3, 'muhtasari': 'Chakula kilikuwa kitamu lakini huduma ilichelewa sana.'}
Hisia: wastani | Alama: 3

Say: "Now we have a Python dict. We can save it to the SQLModel database we learned about!"

Step 2: Tool use — tool_demo.py

# tool_demo.py
import json
from msaidizi import client, maandishi

MODEL = "claude-haiku-5-5"

# "Database" yetu ndogo ya duka
BEI_ZA_DUKA = {"sukari": 3200, "mchele": 2800, "unga": 1800, "mafuta": 4500}  # TSh kwa kilo/lita


def pata_bei(bidhaa: str) -> str:
    bei = BEI_ZA_DUKA.get(bidhaa.lower())
    if bei is None:
        return json.dumps({"kosa": f"Bidhaa '{bidhaa}' haipo dukani"})
    return json.dumps({"bidhaa": bidhaa, "bei_tsh": bei})


TOOLS = [{
    "name": "pata_bei",
    "description": "Pata bei ya sasa ya bidhaa moja kwenye duka la Mama Neema (TSh kwa kilo au lita).",
    "input_schema": {
        "type": "object",
        "properties": {"bidhaa": {"type": "string", "description": "Jina la bidhaa, mf. sukari"}},
        "required": ["bidhaa"],
    },
}]

messages = [{"role": "user", "content": "Nikinunua kilo 2 za sukari na kilo 3 za mchele, nitalipa kiasi gani?"}]

while True:
    response = client.messages.create(
        model=MODEL, max_tokens=2048, tools=TOOLS, messages=messages,
        system="Wewe ni muuzaji wa duka la Mama Neema. Tumia tool pata_bei kupata bei. Jibu kwa Kiswahili.",
    )
    if response.stop_reason != "tool_use":
        break

    messages.append({"role": "assistant", "content": response.content})
    matokeo = []
    for block in response.content:
        if block.type == "tool_use":
            print(f"  [Claude anaomba tool: {block.name}({block.input})]")
            matokeo.append({
                "type": "tool_result",
                "tool_use_id": block.id,
                "content": pata_bei(**block.input),
            })
    messages.append({"role": "user", "content": matokeo})

print(maandishi(response))

Output (approximately):

  [Claude anaomba tool: pata_bei({'bidhaa': 'sukari'})]
  [Claude anaomba tool: pata_bei({'bidhaa': 'mchele'})]
Sukari kilo 2 × 3,200 = 6,400 na mchele kilo 3 × 2,800 = 8,400. Jumla utalipa TSh 14,800.

Explain: "Claude asked for two tools at once. We are the ones who ran them. We return all the results in ONE user message."

Step 3: FastAPI — main.py

# main.py
import json

import anthropic
from dotenv import load_dotenv
from fastapi import FastAPI, HTTPException
from fastapi.responses import StreamingResponse
from pydantic import BaseModel, Field

load_dotenv()
client = anthropic.Anthropic()
MODEL = "claude-haiku-5-5"
SYSTEM = "Wewe ni msaidizi wa Kiswahili. Jibu kwa ufupi na kwa usahihi. Kama hujui, sema hujui."

app = FastAPI(title="AI API ya Kiswahili (Wiki 14)")


class Swali(BaseModel):
    swali: str = Field(min_length=1, max_length=2000)


class Maoni(BaseModel):
    maandishi: str = Field(min_length=1, max_length=3000)


def maandishi(response) -> str:
    return "".join(b.text for b in response.content if b.type == "text")


def ita_claude(**kwargs):
    """Ita Claude na badilisha errors kuwa HTTPException."""
    try:
        return client.messages.create(model=MODEL, **kwargs)
    except anthropic.AuthenticationError:
        raise HTTPException(status_code=500, detail="Tatizo la usanidi wa server (API key).")
    except anthropic.RateLimitError:
        raise HTTPException(status_code=429, detail="Maombi mengi mno. Jaribu tena baadaye.")
    except anthropic.APIConnectionError:
        raise HTTPException(status_code=503, detail="Huduma ya AI haipatikani kwa sasa.")
    except anthropic.APIStatusError as e:
        raise HTTPException(status_code=502, detail=f"Hitilafu ya huduma ya AI ({e.status_code}).")


@app.post("/uliza")
def uliza(s: Swali):
    r = ita_claude(max_tokens=1024, system=SYSTEM,
                   output_config={"effort": "low"},
                   messages=[{"role": "user", "content": s.swali}])
    if r.stop_reason == "refusal":
        raise HTTPException(status_code=422, detail="Samahani, siwezi kujibu ombi hilo.")
    return {
        "jibu": maandishi(r),
        "tokens": {"input": r.usage.input_tokens, "output": r.usage.output_tokens},
    }


@app.post("/uliza-stream")
def uliza_stream(s: Swali):
    def vipande():
        with client.messages.stream(model=MODEL, max_tokens=1024, system=SYSTEM,
                                    output_config={"effort": "low"},
                                    messages=[{"role": "user", "content": s.swali}]) as stream:
            for text in stream.text_stream:
                yield text
    return StreamingResponse(vipande(), media_type="text/plain; charset=utf-8")


SCHEMA_MAONI = {
    "type": "object",
    "properties": {
        "hisia": {"type": "string", "enum": ["chanya", "hasi", "wastani"]},
        "alama": {"type": "integer"},
        "muhtasari": {"type": "string"},
    },
    "required": ["hisia", "alama", "muhtasari"],
    "additionalProperties": False,
}


@app.post("/chambua-maoni")
def chambua_maoni(m: Maoni):
    r = ita_claude(
        max_tokens=1024,
        system="Chambua maoni ya mteja. alama ni 1 hadi 5. Muhtasari kwa Kiswahili, sentensi moja.",
        messages=[{"role": "user", "content": m.maandishi}],
        output_config={"format": {"type": "json_schema", "schema": SCHEMA_MAONI}},
    )
    return json.loads(maandishi(r))

uvicorn main:app --reload

Try it at http://127.0.0.1:8000/docs, or in PowerShell:

Invoke-RestMethod -Uri http://127.0.0.1:8000/uliza -Method Post -ContentType "application/json; charset=utf-8" -Body '{"swali": "Mji mkuu wa Tanzania ni upi?"}'

Output:

jibu                                   tokens
----                                   ------
Mji mkuu wa Tanzania ni Dodoma.        @{input=40; output=12}

To test streaming, create mteja_stream.py (stream client; in another terminal, with the venv activated):

# mteja_stream.py
import httpx

with httpx.stream("POST", "http://127.0.0.1:8000/uliza-stream",
                  json={"swali": "Nieleze mzunguko wa maji kwa aya moja."}, timeout=60) as r:
    for kipande in r.iter_text():
        print(kipande, end="", flush=True)
print()

Step 4: Tests without spending money — test_main.py

# test_main.py
import json
from types import SimpleNamespace

from fastapi.testclient import TestClient

import main


def jibu_bandia(text: str):
    """Tengeneza object inayofanana na jibu la Claude."""
    return SimpleNamespace(
        content=[SimpleNamespace(type="text", text=text)],
        stop_reason="end_turn",
        usage=SimpleNamespace(input_tokens=10, output_tokens=5),
    )


class ClientBandia:
    def __init__(self, text):
        self.messages = SimpleNamespace(create=lambda **kwargs: jibu_bandia(text))


def test_uliza(monkeypatch):
    monkeypatch.setattr(main, "client", ClientBandia("Dodoma"))
    r = TestClient(main.app).post("/uliza", json={"swali": "Mji mkuu?"})
    assert r.status_code == 200
    assert r.json()["jibu"] == "Dodoma"
    assert r.json()["tokens"] == {"input": 10, "output": 5}


def test_swali_tupu_linakataliwa():
    r = TestClient(main.app).post("/uliza", json={"swali": ""})
    assert r.status_code == 422


def test_chambua_maoni(monkeypatch):
    data = {"hisia": "chanya", "alama": 5, "muhtasari": "Mteja ameridhika."}
    monkeypatch.setattr(main, "client", ClientBandia(json.dumps(data)))
    r = TestClient(main.app).post("/chambua-maoni", json={"maandishi": "Nimefurahi sana!"})
    assert r.status_code == 200
    assert r.json()["hisia"] == "chanya"

pytest -v

Say: "These tests use neither internet nor money, and they pass every time. We are testing OUR code, not Claude."

Step 5 (optional, just show it): Ollama, a free alternative

If you have time and the computer has RAM ≥ 8GB: install Ollama from https://ollama.com/download, then:

ollama pull llama3.2
ollama run llama3.2 "Habari, jina lako nani?"

For Python: pip install ollama then ollama.chat(model="llama3.2", messages=[{"role": "user", "content": "..."}]). Tell the student: it is free and the data doesn't leave the computer, but it is slower and its Swahili is weaker.

For those who want more (for the Sonnet model only; mention it, don't stress it): For claude-sonnet-5-5, Anthropic recommends turning on the server "fallback": if a request is refused for safety reasons (rarely, even for legitimate questions), the API automatically tries another model:

r = client.beta.messages.create(
    model="claude-sonnet-5-5",
    max_tokens=2048,
    betas=["server-side-fallback-2026-07-01"],
    fallbacks="default",
    messages=[{"role": "user", "content": "Eleza jinsi chanjo zinavyofanya kazi."}],
)
print(r.stop_reason, maandishi(r))

Questions to ask the student

1. Question: Who runs the pata_bei function: Claude or us? Answer: Us. Claude only asks through a tool_use block.
2. Question: What is tool_use_id used for? Answer: To link each tool_result with the corresponding tool request.
3. Question: The advantage of structured output over just saying "answer in JSON"? Answer: It guarantees valid JSON matching the schema; no json.loads failures.
4. Question: Why max_length=2000 on Swali (question)? Answer: To prevent oversized requests that increase cost, or misuse.
5. Question: Why do tests use a fake client? Answer: They use no money/internet, are fast, and the answers don't change.
6. Question: Pros and cons of Ollama? Answer: Pros: free, private, no internet. Cons: needs a powerful computer, slow, lower quality (especially Swahili).

Common mistakes

In-class exercise

Exercise 3.1: Add an endpoint POST /tafsiri (translate) that receives {"maandishi": "...", "lugha": "Kiingereza"} (text, language) and returns {"tafsiri": "..."}. lugha must be one of: Kiswahili, Kiingereza, Kifaransa (Swahili, English, French; use Literal).

Exercise 3.2: Add a second tool ongeza_bidhaa_kwenye_kikapu(bidhaa, kilo) (add product to cart) to tool_demo.py that saves to a list KIKAPU (cart), then ask: "Niwekee kilo 2 za unga na lita 1 ya mafuta kwenye kikapu, kisha niambie jumla." (Put 2 kilos of flour and 1 litre of oil in the cart for me, then tell me the total.)

Exercise 3.3: Write a test of /tafsiri with a fake client.

# 3.1 — ongeza kwenye main.py
from typing import Literal

class Tafsiri(BaseModel):
    maandishi: str = Field(min_length=1, max_length=2000)
    lugha: Literal["Kiswahili", "Kiingereza", "Kifaransa"]


@app.post("/tafsiri")
def tafsiri(t: Tafsiri):
    r = ita_claude(
        max_tokens=1024,
        system="Wewe ni mtafsiri. Rudisha tafsiri tu, bila maelezo ya ziada.",
        output_config={"effort": "low"},
        messages=[{"role": "user", "content": f"Tafsiri kwenda {t.lugha}:\n\n{t.maandishi}"}],
    )
    return {"tafsiri": maandishi(r).strip()}

# 3.2 — tool_demo2.py
import json
from msaidizi import client, maandishi

BEI_ZA_DUKA = {"sukari": 3200, "mchele": 2800, "unga": 1800, "mafuta": 4500}
KIKAPU = []


def pata_bei(bidhaa: str) -> str:
    bei = BEI_ZA_DUKA.get(bidhaa.lower())
    return json.dumps({"bidhaa": bidhaa, "bei_tsh": bei} if bei else {"kosa": "haipo"})


def ongeza_bidhaa_kwenye_kikapu(bidhaa: str, kilo: float) -> str:
    if bidhaa.lower() not in BEI_ZA_DUKA:
        return json.dumps({"kosa": f"{bidhaa} haipo dukani"})
    KIKAPU.append({"bidhaa": bidhaa.lower(), "kilo": kilo})
    return json.dumps({"sawa": True, "kikapu": KIKAPU})


FUNCTIONS = {"pata_bei": pata_bei, "ongeza_bidhaa_kwenye_kikapu": ongeza_bidhaa_kwenye_kikapu}

TOOLS = [
    {"name": "pata_bei", "description": "Pata bei ya bidhaa (TSh kwa kilo/lita).",
     "input_schema": {"type": "object", "properties": {"bidhaa": {"type": "string"}},
                      "required": ["bidhaa"]}},
    {"name": "ongeza_bidhaa_kwenye_kikapu", "description": "Weka bidhaa kwenye kikapu cha mteja.",
     "input_schema": {"type": "object",
                      "properties": {"bidhaa": {"type": "string"},
                                     "kilo": {"type": "number", "description": "kilo au lita"}},
                      "required": ["bidhaa", "kilo"]}},
]

messages = [{"role": "user", "content":
             "Niwekee kilo 2 za unga na lita 1 ya mafuta kwenye kikapu, kisha niambie jumla."}]
while True:
    r = client.messages.create(model="claude-haiku-5-5", max_tokens=2048, tools=TOOLS,
                               messages=messages,
                               system="Wewe ni muuzaji wa duka. Tumia tools. Jibu kwa Kiswahili.")
    if r.stop_reason != "tool_use":
        break
    messages.append({"role": "assistant", "content": r.content})
    messages.append({"role": "user", "content": [
        {"type": "tool_result", "tool_use_id": b.id, "content": FUNCTIONS[b.name](**b.input)}
        for b in r.content if b.type == "tool_use"
    ]})

print(maandishi(r))
print("KIKAPU:", KIKAPU)   # unga 2 (3,600) + mafuta 1 (4,500) = 8,100

# 3.3 — ongeza kwenye test_main.py
def test_tafsiri(monkeypatch):
    monkeypatch.setattr(main, "client", ClientBandia("Good morning"))
    r = TestClient(main.app).post("/tafsiri", json={"maandishi": "Habari za asubuhi", "lugha": "Kiingereza"})
    assert r.status_code == 200
    assert r.json() == {"tafsiri": "Good morning"}


def test_tafsiri_lugha_isiyokubalika():
    r = TestClient(main.app).post("/tafsiri", json={"maandishi": "Habari", "lugha": "Kichina"})
    assert r.status_code == 422

Homework

(Due: next session)

Mini project: "Recipe Assistant"

1. CLI mapishi.py (recipes): a chatbot with a system prompt of "Chef of Tanzanian food", using streaming and history, with the commands toka (exit), upya (restart), gharama (cost).

2. FastAPI POST /mapishi that receives {"viungo": ["mchele", "nazi", "samaki"]} (ingredients: rice, coconut, fish; a non-empty list, at most 15 items) and returns JSON with this structure using structured output:

   json

   Copy

   {"jina_la_chakula": "...", "muda_dakika": 40, "hatua": ["...", "..."]}
   

3. 2 pytest tests (with a fake client): one for a correct response, one for an empty list being rejected (422).

4. Make sure .env is NOT in git status, and that there is a .env.example. Commit.

# mapishi.py
from dotenv import load_dotenv
import anthropic

load_dotenv()
client = anthropic.Anthropic()
MODEL = "claude-haiku-5-5"
SYSTEM = ("Wewe ni mpishi hodari wa vyakula vya Kitanzania (pilau, ugali, wali wa nazi, "
          "mishkaki, chapati...). Jibu kwa Kiswahili, kwa hatua zilizo wazi na fupi.")


def main():
    historia, t_in, t_out = [], 0, 0
    while True:
        u = input("Wewe: ").strip()
        if not u:
            continue
        if u == "toka":
            break
        if u == "upya":
            historia = []
            continue
        if u == "gharama":
            usd = (t_in * 0.10 + t_out * 0.50) / 1_000_000
            print(f"${usd:.6f} ≈ TSh {usd * 2600:.2f}")
            continue
        historia.append({"role": "user", "content": u})
        print("Mpishi: ", end="", flush=True)
        with client.messages.stream(model=MODEL, max_tokens=2048, system=SYSTEM,
                                    messages=historia) as s:
            for t in s.text_stream:
                print(t, end="", flush=True)
            f = s.get_final_message()
        print("\n")
        historia.append({"role": "assistant", "content": f.content})
        t_in += f.usage.input_tokens
        t_out += f.usage.output_tokens


if __name__ == "__main__":
    main()

# api_mapishi.py   ->  uvicorn api_mapishi:app --reload
import json
import anthropic
from dotenv import load_dotenv
from fastapi import FastAPI
from pydantic import BaseModel, Field

load_dotenv()
client = anthropic.Anthropic()
app = FastAPI()

SCHEMA = {
    "type": "object",
    "properties": {
        "jina_la_chakula": {"type": "string"},
        "muda_dakika": {"type": "integer"},
        "hatua": {"type": "array", "items": {"type": "string"}},
    },
    "required": ["jina_la_chakula", "muda_dakika", "hatua"],
    "additionalProperties": False,
}


class Viungo(BaseModel):
    viungo: list[str] = Field(min_length=1, max_length=15)


@app.post("/mapishi")
def mapishi(v: Viungo):
    r = client.messages.create(
        model="claude-haiku-5-5",
        max_tokens=2048,
        system="Pendekeza chakula kimoja cha Kitanzania kwa viungo ulivyopewa. Kwa Kiswahili.",
        messages=[{"role": "user", "content": "Viungo: " + ", ".join(v.viungo)}],
        output_config={"format": {"type": "json_schema", "schema": SCHEMA}},
    )
    text = "".join(b.text for b in r.content if b.type == "text")
    return json.loads(text)

# test_api_mapishi.py
import json
from types import SimpleNamespace
from fastapi.testclient import TestClient
import api_mapishi


def test_mapishi(monkeypatch):
    data = {"jina_la_chakula": "Wali wa nazi na samaki", "muda_dakika": 45, "hatua": ["Osha mchele"]}
    jibu = SimpleNamespace(content=[SimpleNamespace(type="text", text=json.dumps(data))])
    bandia = SimpleNamespace(messages=SimpleNamespace(create=lambda **kw: jibu))
    monkeypatch.setattr(api_mapishi, "client", bandia)
    r = TestClient(api_mapishi.app).post("/mapishi", json={"viungo": ["mchele", "nazi", "samaki"]})
    assert r.status_code == 200
    assert r.json()["muda_dakika"] == 45


def test_viungo_tupu():
    r = TestClient(api_mapishi.app).post("/mapishi", json={"viungo": []})
    assert r.status_code == 422

If you are asked a question you don't know

Say: "That's a good question; this technology changes very fast, let's check the official documentation together." Search: "Claude API python", or the full error message. Don't invent parameter names: check the docs.

• Claude API docs: https://platform.claude.com/docs/en/api/overview
• Python SDK (GitHub): https://github.com/anthropics/anthropic-sdk-python
• Prompt engineering: https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/overview

End-of-week checklist

• Explains what an LLM is and what a token is, and why Swahili may have more tokens
• API key is in .env, .env is in .gitignore, .env.example exists, and the key hasn't been shown on screen
• Calls client.messages.create and extracts text by filtering for "text" blocks
• Reads stop_reason and usage, and calculates the cost
• Catches errors (AuthenticationError, RateLimitError, APIConnectionError)
• Uses a system prompt and explains how it differs from a user message
• Their CLI chatbot remembers history and uses streaming
• Explains the tool use cycle (tool_use → we run it → tool_result)
• Gets structured JSON with output_config + schema
• Has a working FastAPI endpoint /uliza, and tests that use a fake client
• Knows 3 techniques for reducing cost and knows what Ollama is