From student to AI Engineer.

What is RAG, and why

RAG stands for Retrieval Augmented Generation. Instead of relying on the model's general knowledge, you retrieve relevant information from your own data and pass it to the model as context, so it answers from what you gave it. Every RAG system is three pieces working together: a data source, a retrieval system, and a language model.

A few things people build with it:

• A chatbot that answers from a company's internal documents
• A support assistant scoped to your product's docs
• A legal tool that searches contracts and summarizes findings

Embeddings

You just saw that RAG works by retrieving the pieces of your data that are relevant to a question. But there is a catch: computers don't understand words the way we do. To a machine, "car" and "automobile" are just two unrelated strings of letters, so how could it ever tell which pieces are relevant?

That is the problem embeddings solve. Text is turned into a list of numbers (a vector) that captures its meaning, so pieces of text that mean similar things land close together. That closeness is exactly what searching "by meaning" (semantic search) measures. The video below explains it with single words; in RAG the same trick is applied to whole sentences and document chunks.

Preparing your data: loading and chunking

So meaning lives in embeddings, and the text you embed is your own. But you can't embed an 80-page PDF in one shot. First you get the raw data in (PDFs, Word docs, web pages, databases, and it rarely comes clean), strip the noise, then split it into chunks: pieces small enough to embed and retrieve precisely. How you chunk decides what the model can and can't find later, so it matters more than it looks.

Storing and searching: vector databases

Now every chunk has an embedding. Where do thousands of these vectors go, and how do you search them in an instant? That is the job of a vector database. It stores all those embeddings and finds the closest ones to a query by meaning, not exact keywords. Popular options (Pinecone, Chroma, Weaviate, FAISS) trade off cost, hosting, and ease of use. You index your chunks once, then run a similarity search on every question to pull the most relevant ones.

Build the full pipeline

You now have the three pieces from step 1: your data (prepared and chunked), a retrieval system (embeddings in a vector database), and the model. The pipeline is the loop that ties them together on every question, and its name spells out the three steps:

• Retrieval, turn the question into an embedding and find the closest chunks
• Augmentation, add those chunks to the prompt as context
• Generation, the model reads them and answers, grounded in what came back

Because the answer is built from real documents, tell the model to use only the provided context, so every answer traces back to a source.

One thing to watch: retrieval quality is the biggest lever. If the wrong chunks come back, the answer is wrong no matter how capable the model is, which is why the re-ranking and query techniques in the playlist matter. The playlist builds all of this in LangChain, the most widely used framework (LlamaIndex is the main alternative, strong for document-heavy apps).

Evaluating your RAG system

You have a working pipeline. But how do you know it is actually any good? A RAG system can fail in two places, so you check two layers. Retrieval: did the right chunks come back for the question? Answer faithfulness: is the response actually grounded in those chunks, or did the model add things? (A fluent, confident answer can still be wrong.)

Keep a small test set of questions paired with the passage that should answer them, and use an LLM-as-judge to score faithfulness and relevance across many questions at once, so you can tell whether a change helped or hurt.

What is an agent

An agent is not a prompt, it is an architecture. A well-designed agent decides what to do next, uses tools to get information or take actions, checks its own work, and keeps going until the task is done. A prompt tells the AI what to do once; an agent figures out how to do it across multiple steps.

The agent loop

So an agent works across multiple steps, but how does it decide each one? Underneath every agent is a simple cycle that repeats until the task is done. This loop is the key mental model for everything else in this stage.

• Reason, the model decides what needs to happen next
• Act, it takes an action, usually by calling a tool
• Observe, it reads the result of that action
• Repeat, it uses what it observed to decide the next step, stopping once complete

Core agent components

That loop needs machinery to run. Every agent is built from four parts:

• The model, the reasoning engine that drives every decision
• Tools, how the agent acts: APIs, search engines, databases, code, even other models
• Memory, short-term (this conversation) and long-term (across sessions)
• Error handling, when an action fails it recognizes the failure, adjusts, and tries a different approach

Agentic patterns

With those parts in place, a few designs for how agents think and act keep recurring. You'll see these four everywhere.

• Reflection, the agent reviews its own output and revises before returning it
• Tool use, letting the model decide when to reach for a tool rather than answering from memory
• Planning, breaking a complex task into ordered steps
• Multi-agent collaboration, specialized agents working together toward a shared goal; this is where things get genuinely powerful

Evaluating your agent

Once your agent runs, how do you know it works? This is the hardest thing to evaluate, because you're judging a whole sequence of decisions, not a single answer.

• Task success, did it accomplish the goal end to end? The measure that matters most
• Trajectory, did it take a sensible path with the right tools in a reasonable order?
• Efficiency, how many steps, tool calls, and tokens did it take?
• Agent test cases + LLM-as-judge, score both the result and the steps

When to use an agent

Before you commit to all this, a check: not every problem needs an agent. If a task is a fixed, predictable sequence, a simple workflow is cheaper, faster, and more reliable. Reach for an agent only when the steps can't be known in advance. Knowing when not to build one is part of the skill.

Frameworks

You won't wire the loop by hand forever. Frameworks handle the plumbing so you can focus on the logic.

• LangChain, the most widely used framework, a great starting point
• LangGraph, built on LangChain for multi-step and multi-agent workflows
• MCP (Model Context Protocol), a growing standard for connecting agents to external tools

Always start with the official documentation. The people who built the tool are the best teachers.

FastAPI

Before you deploy anything, wrap it in an API so other systems and users can interact with it. FastAPI is the most popular Python framework for this, and the one most AI engineers reach for first.

• Building API endpoints
• Handling requests and responses
• Adding basic authentication
• Documenting your API automatically

Docker

Your API runs on your laptop; now make it run anywhere. Docker packages your entire application, code, dependencies and configuration, into one container that behaves the same on every machine. No more "it works on my machine."

• What containerization means and why it matters
• Writing a Dockerfile
• Building and running containers
• Deploying a containerized application

Cloud

A container still needs somewhere to run around the clock; that somewhere is the cloud. There are two moves here: get your first project live fast on a beginner-friendly platform, and, separately, learn one big cloud deeply for your career. The concepts transfer, so you only ever need to go deep on one.

Token cost awareness

You are live; now keep it from quietly draining money. Every call to an LLM API costs money based on how many tokens you send and receive. Think about this from day one.

• How token pricing works
• How to estimate the cost of your application
• Strategies to keep costs under control without hurting quality