Building your first RAG system

Let's build the simplest thing that works, so the rest of the course has something to improve. In pseudocode, a minimal RAG system is short:

# INDEX (once)
chunks = split(load(documents), size=500, overlap=75)
index = vector_db()
for c in chunks:
    index.add(embed(c), metadata={"text": c, "source": c.source})

# QUERY (each question)
def answer(question):
    hits = index.search(embed(question), k=5)     # retrieve
    context = "\n\n".join(h.text for h in hits)
    prompt = f"Answer using ONLY the context. Cite sources. If not in context, say so.\n\nContext:\n{context}\n\nQ: {question}"
    return model.generate(prompt)

That's a real RAG system. You chunk your documents, embed and store them, and at query time you retrieve the closest chunks, stuff them in the prompt with a grounding instruction ("use only the context; cite; abstain if absent"), and generate. Frameworks like LangChain or LlamaIndex give you these pieces prebuilt, and a vector store like pgvector (if you already run Postgres), Qdrant, or Pinecone handles the index.

Notice the grounding instruction — it's the same "cite or abstain" discipline from the enterprise course, now applied automatically. Without it, the model may ignore your context and free-associate; with it, you get traceable, honest answers.

Now run it and watch it fail, because those failures motivate everything ahead. Ask a question whose answer spans two chunks that got split apart, and it misses. Search for an exact error code or product name, and pure semantic search returns vaguely-related fluff instead of the exact match. Ask something your docs don't cover, and a weak grounding prompt lets it hallucinate anyway. Feed it fifty chunks and quality drops as the real answer gets buried.

Every one of those is a solved problem — better chunking, hybrid search, reranking, contextual retrieval, evaluation. That's the rest of the course. But you have to feel naive RAG break first to understand why each technique exists. Build this, point it at a handful of your own documents, and find its breaking points before we fix them.