DEV Community: Olha Piliaieva

AWS Bedrock Converse Stream Tutorial. Build a Python Terminal Chatbot

Olha Piliaieva — Wed, 04 Nov 2026 21:45:22 +0000

Same chatbot, same question, I asked twice: "What did I say I bought?"

First run, it answers immediately. Headphones, last week. Second run, it has no idea and politely asks me which product I mean.

Nothing changed on the AWS side between those two runs. Same model, same region, same prompt. I changed two lines of my own Python, and that is the entire subject of this post, because those two lines are how every chatbot you have ever used "remembers" you.

I'll show you a support bot for my test shop BrightCart, which is a made-up online electronics store I use across this AWS series so I don't have to keep inventing products. It runs in my terminal, it talks to Amazon Bedrock, and it does three things a single API call doesn't: follows a system prompt, so it knows who it is and how long its answers should be, remembers the conversation and streams the answer as it's written instead of leaving you looking at an empty line.

Model is Amazon Nova 2 Lite. Cheap, fast, and for a support bot answering short questions all day that's the whole shopping list.

Before you start
Python, boto3, an AWS account with Amazon Bedrock available in us-east-1, and credentials that work in your terminal. I use aws login. If none of that is set up yet, the earlier post walks through it: Building GenAI apps and AI agents on AWS: a hands-on path

Code lives in the repo https://github.com/Foxfix/agenticai_mini_course.git

The system prompt

Instructions that sit above the whole conversation. The customer never sees them, the model reads them before every single answer it writes.

Ours is one sentence: you are BrightCart's friendly support assistant, answer in three short sentences or less. That length limit is doing real work, not decoration. Support answers should be short because people are annoyed when they're reading them, and short answers also cost less, since you pay for every token the model produces.

In the Converse API, which is the one request format every Bedrock chat model accepts, the system prompt gets its own field called system, sitting right next to messages. You could paste the instructions onto the front of the first user message instead and it would mostly work. I wouldn't do it. Separate field tells the model "this is policy". Glued onto a user message, it tells the model "a customer said this", and eventually a customer will say something that argues with it.

Memory is a list. That's the whole trick.

Here's the surprise, and it catches almost everyone: the model remembers nothing. Every request starts from zero. There is no session on the Bedrock side quietly holding your conversation, there is nothing to attach to or expire.

So how does any chatbot remember what you said five messages ago? Your code keeps a list: each turn you add the customer’s message, send the whole list back, and add the model’s answer to it, and that last step is the one people drop.

If you skip it, the list only ever contains what the customer said. The bot can see every question it was asked and none of the answers it gave, which produces a very specific and very confusing kind of broken: it repeats itself, it re-introduces itself, it offers you the same three options it already offered. It looks like the model has gone stupid. The model is fine. It's reading a transcript with half the dialogue missing.

Next turn the list is longer, the model sees everything that was said, and that is the memory. All of it. There is no second mechanism.

One side effect to know now rather than later. You resend the entire conversation on every request, so input tokens climb with every message, and a long chat gets more expensive as it goes. Real products trim old messages or summarise them for exactly this reason. For a terminal demo it's irrelevant. The day you put a bot in front of actual customers it stops being irrelevant.

Streaming

A normal converse call makes you wait until the whole answer exists, then hands it over in one piece. converse_stream sends the answer in fragments while the model is still writing, and your code prints each fragment the moment it lands.

Total time is roughly the same. It just feels enormously faster, because the first words appear almost instantly instead of after two seconds of nothing. For anything a human is sitting in front of, that difference is worth the extra six lines of code.

"""A03 - a support chatbot in the terminal: system prompt, memory of the conversation, streaming."""
import sys
sys.path.insert(0, __file__.rsplit("/", 2)[0])
import boto3
from config import REGION, MODEL_ID

bedrock = boto3.client("bedrock-runtime", region_name=REGION)
SYSTEM = [{"text": "You are the friendly support assistant of BrightCart, an online electronics store. Answer in 3 short sentences or less."}]
history = []

print("BrightCart support (type 'quit' to exit)")
while True:
    question = input("\nYou: ").strip()
    if question.lower() in ("quit", "exit"):
        break
    history.append({"role": "user", "content": [{"text": question}]})

    stream = bedrock.converse_stream(modelId=MODEL_ID, system=SYSTEM, messages=history,
                                     inferenceConfig={"maxTokens": 400, "temperature": 0.3})
    print("Bot: ", end="", flush=True)
    answer = ""
    for event in stream["stream"]:
        if "contentBlockDelta" in event:
            piece = event["contentBlockDelta"]["delta"].get("text", "")
            answer += piece
            print(piece, end="", flush=True)
    print()
    history.append({"role": "assistant", "content": [{"text": answer}]})

REGION and MODEL_ID come out of a small config.py in the repo. MODEL_ID defaults to Nova 2 Lite and you can override it with an environment variable, which is handy when you want to try another model without editing anything.

SYSTEM is a list of content

blocks, same shape as message content. And history = [] is the bot's entire memory. One empty list. That's the thing the title of this post is about.

Inside the loop the message goes into history, converse_stream gets the system prompt and the whole list, and each contentBlockDelta prints the moment it arrives. The last line appends the answer back into history, and we’re about to delete it.

I ran it, told it about the headphones, then asked what I’d said I bought, and it answered correctly, because the list remembered.
python genai/a03_chat.py

I bought headphones last week, they stopped working, what are my options?
What did I say I bought? It remembers.

Now let's break it

Claiming the memory lives in that list is easy. Proving it takes two edits and about thirty seconds, and it's the most useful thirty seconds in this whole post, because a thing you have broken on purpose is a thing you understand.

Two changes. Comment out the last line, the one that appends the bot's answer. And instead of appending the customer's message to the history, replace the history with that one message:

    history = [{"role": "user", "content": [{"text": question}]}]
    ...
    # history.append({"role": "assistant", "content": [{"text": answer}]})

Now every request carries exactly one message and nothing else. The rest of the code remained completely untouched.
Run it again and paste the same two messages. Then the test question, and it has no clue. Same model, same settings, no memory, because the memory was never on the AWS side at all, it was that list.

Then I put it back.

When it doesn't work

The bot forgets. One of those two lines. Print len(history) in the loop and watch whether it grows.

Streaming doesn't stream. Missing flush=True.

Empty chunks. Not every delta carries text, that's what .get("text", "") is for.

Two answers at once. You pasted two lines.

AccessDeniedException. Expired credentials (aws login lasts 12 hours) or the wrong region.

Full code: github.com/Foxfix/agenticai_mini_course

I publish hands-on material on AWS, AI and deployment at @CodeAiFlow. I'm also an AWS Community Builder.

Stop Managing GPUs. Use the AWS Bedrock Converse API - One Python API, Hundreds of LLMs

Olha Piliaieva — Wed, 28 Oct 2026 11:23:36 +0000

If you’ve ever tried adding an AI feature to an app, you know that you should set up a server, figure out GPU hosting, and spend half your week reading infrastructure docs. Cloud providers finally realized nobody wants to do that.
Let's get acquainted with Amazon Bedrock. Instead of managing servers, you get a single API endpoint that connects to hundreds of foundation models, like Claude, Llama, Nova, Mistral, and more.
No infrastructure: No servers or GPUs to configure.

Pay-per-token. You pay only for what you use.
Prompts and responses are not used to train models and are not shared with providers.
Inside you can find lists of over 200 serverless and marketplace models. You can filter by provider (Amazon Nova, Anthropic Claude, Meta Llama, etc.) to check parameters and capabilities.
Playground lets you test prompts in your browser before writing code. Pick a lightweight option like Amazon Nova 2 Light, give it a system prompt, and run a quick test.
Some models (like Meta Llama) have regional restrictions or licensing limits. Providers like Anthropic require a one-time use-case form per account (company name, website, industry, and use-case description) before granting model access.
Bedrock uses the Converse API, providing a uniform request structure across all models. The payload requires three main fields: modelId , messages , inferenceConfig .

import boto3
from config import REGION, MODEL_ID

bedrock = boto3.client("bedrock-runtime", region_name=REGION)

start = time.time()
response = bedrock.converse(
    modelId=MODEL_ID,
    messages=[{"role": "user", "content": [{"text": "Write a one-line welcome message for BrightCart, an online electronics store."}]}],
    inferenceConfig={"maxTokens": 200, "temperature": 0.5},
)
seconds = time.time() - start

print("Model:", MODEL_ID)
print("Answer:", response["output"]["message"]["content"][0]["text"])
usage = response["usage"]
print(f"Tokens: {usage['inputTokens']} in, {usage['outputTokens']} out · {seconds:.1f} s")

In config.py

REGION = "us-east-1"

# Model IDs (copy the newest ones from Bedrock console > Model catalog > model card > "Inference profile ID")
NOVA = "us.amazon.nova-2-lite-v1:0"
CLAUDE = "global.anthropic.claude-sonnet-4-5-20250929-v1:0"

MODEL_ID = os.getenv("MODEL_ID", NOVA)

Two Metrics we have to monitor
Token Breakdown: Models process fragments of words called tokens, not raw text. You pay for both input tokens (your prompt) and output tokens (the generated response). Different models split text differently-the same prompt might cost 60 tokens on one model and 25 on another. Keep an eye on the usage stats returned in the API payload.

Temperature Setting: Controls randomness.

Low (0.1-0.3): Keeps outputs deterministic and grounded. Perfect for customer support, document parsing, or factual Q&A.

High (0.7-1.0): Adds variety and creative range. use for marketing copy or creative brainstorming.

By using AWS Bedrock in combination with Python and Boto3, you gain a convenient and scalable way to run advanced large language models (LLMs) within your own infrastructure-without the need to configure servers or lose control over costs.

Source code for all hands-on episodes: GitHub repository

I’ll be sharing plenty of hands-on videos about AWS, AI and deployment on my YouTube channel, @CodeAiFlow every week. This article follows the second episode of the series, if you'd rather watch the setup step by step. I'm also an AWS Community Builder: my profile on AWS Builder Center.

Building GenAI apps and AI agents on AWS: a hands-on path

Olha Piliaieva — Tue, 20 Oct 2026 15:41:25 +0000

This is the first practical article in the “GenAI and AI agents on AWS” series, and we start with the account setup.

Quick definition so we're on the same page: an agent is an LLM in a while loop. It picks an action, calls a tool, looks at what came back, picks the next one. It stops when it thinks it's done, or when you stop it, which is where the budget alarm comes in.

The model never runs anything itself. It reads text and writes text, your code does the rest.

We're building agents on AWS. Amazon Bedrock gives us the models, Strands Agents is what we write the agent in, Amazon Bedrock AgentCore runs the result in the cloud. Every trip around that loop is a paid request, so the account comes first. Four things before any code: an IAM user, a budget alarm, the local tools, and a login from the terminal without access keys.

IAM user, not root

Root owns everything, billing and closing the account included. Keep it for those rare jobs. Day to day you want an IAM user, and some services refuse root anyway: you can't create a Bedrock Knowledge Base with it.

Create the user: IAM → Users → Create user. I called mine ai-series, call yours whatever. Tick console access and set your own password. One checkbox to untick: "User must create a new password at next sign-in". It's your user, you just typed the password, forcing a change straight away is pointless.

Permissions: attach policies directly, and you need two. AdministratorAccess (fine for learning, in a real company you'd give far less) and SignInLocalDevelopmentAccess, the one that lets the CLI log in through the browser. That second one is why we won't need access keys later.

After you click create you get a page with a console sign-in link for your account. Copy it. It has your account ID in it, so keep it out of screenshots. Sign out of root and open that link. The console says Europe (Stockholm) for me. Ignore it, IAM users are global. We'll switch to us-east-1 when we build.

Budget alarm

AI services charge per request, so a loop gone wrong can cost real money. Type "budgets" in the console search, it takes you to Billing and Cost Management. Budgets → Create budget, keep "Use a template", pick Monthly cost budget. Keep the name. The amount defaults to $100, I changed it to $35. Pick whatever you're comfortable spending in a month, then add your email.

The template sends an alert when actual spend reaches 85% of the budget, again at 100%, and when the forecast says you're going to cross the line. Click Create budget and that's it.

Local tools

Everything here runs inside the project folder.

uv first. It's a fast Python package manager and the AgentCore tool needs it. On a Mac, brew install uv.

Then a virtual environment, so the project's packages stay separate from the rest of your machine, and the requirements:

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

If you see (.venv) at the start of the line, it's on. The requirements bring in boto3, the AWS library for Python, and Strands. The download takes about 20 seconds.

Next the AgentCore CLI. It comes from npm, and I install it into a local tools folder inside the project. This one takes a couple of minutes. Add that folder to PATH and check the version: 0.31 for me. The first run prints a note about usage telemetry. The tool sends anonymous stats to AWS, meaning which commands you run and whether they work. Not your code, not your data. It's on by default and I turn it off.

Last, the AWS CLI. Mine is 2.34 (aws --version), anything from 2.32 works with aws login.

Sign in from the terminal

The old way is access keys: a secret saved in a file on your laptop. It never expires and it leaks surprisingly often. The new way is aws login. The terminal opens the browser, you sign in to the console, and the CLI gets temporary credentials that refresh on their own for up to 12 hours. Nothing secret to copy or lose.

aws login --profile ai-series

The browser opens by itself and AWS asks which console session to share with the command line. Pick the active ai-series one. Back in the terminal, make that profile the default and set the region:

export AWS_PROFILE=ai-series
aws configure set region us-east-1
aws configure get region

The last command should print us-east-1. Done. Not a single key on this laptop.

I’ve practically demonstrated how to do this in my video

Source code for all hands-on episodes: GitHub repository

I’ll be sharing plenty of hands-on videos about AWS, AI and deployment on my YouTube channel, @CodeAiFlow every week. This article follows the first episode of the series, if you'd rather watch the setup step by step. I'm also an AWS Community Builder: my profile on AWS Builder Center.