iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
A basic retrieval-augmented generation (RAG) pipeline answers questions about a bounded set of documents by first retrieving relevant passages, then giving those passages to a language model as evidence. It has two parts: an indexing workflow that prepares documents ahead of time, and a query workflow that retrieves evidence and generates an answer when someone asks a question.
What a beginner RAG pipeline does
RAG combines search with text generation. Instead of asking a model to answer only from its training, the application searches a collection you provide and includes the most relevant excerpts in the prompt. The model can then answer from that context, and your application can show the excerpts so a person can check the answer.
A vector database is only one possible part of this system. A working pipeline also needs document loading, chunking, embeddings, retrieval, prompt construction, model generation, and a way to assess whether answers are supported. LlamaIndex describes RAG in terms of loading, indexing, retrieval, generation, and evaluation stages (LlamaIndex’s RAG overview).
Separate indexing from question answering
| Workflow | When it runs | What it does |
|---|---|---|
| Indexing | When documents are added or changed | Loads source files, divides their text into chunks, creates an embedding for each chunk, and stores the chunks and embeddings for later search. |
| Question answering | For each user question | Embeds the question, retrieves relevant chunks, supplies them and the question to a language model, and returns an answer with its supporting passages. |
Frameworks can combine or hide individual steps, but understanding the distinction helps with updates and debugging. If a source document changes, refresh its indexed content; a new question generally needs retrieval and generation, not a complete re-index of every unchanged file. The broad flow—load, split, embed, index, retrieve, and generate—is also reflected in the provider and framework documentation for OpenAI retrieval, LangChain’s RAG tutorial, and LlamaIndex’s overview.
#1 Best Overall
Choose a small, clear document collection
Start with a bounded set of text documents that you are allowed to process, such as a few Markdown files or plain-text exports. This tutorial uses a local directory called docs/ as the conceptual source. The examples below describe the operations to implement; they are framework-neutral pseudocode, not a copy-and-run program for a particular library. Exact package names, APIs, and setup commands depend on the framework, embedding provider, and storage backend you select.
Before indexing, confirm that the files contain extractable text. Scanned PDFs or documents with complex layouts may need text extraction or OCR before chunking. Remove irrelevant boilerplate where appropriate, and attach useful metadata such as a filename, section, or page number so a retrieved passage can be traced to its origin.
Build the indexing workflow
- Load and normalize. Read each document and preserve its source identity. Clean only what is necessary; stripping headings or list structure can remove context that helps both retrieval and the model.
- Split into chunks. Divide text into passages small enough to retrieve selectively, while retaining enough surrounding context for meaning. Keep metadata with each passage.
- Create embeddings. Convert each chunk into a numerical representation using an embedding model. Store the embedding alongside the chunk text and metadata.
- Index and persist. Make the chunks searchable in a vector store or another retrieval system. Persist the index if it must be reused across application runs.
Chunk size and overlap are configuration choices, not universal constants. For its vector-store file-indexing flow, OpenAI’s retrieval guide currently specifies defaults of max_chunk_size_tokens 800 and chunk_overlap_tokens 400. Those are OpenAI-specific defaults, not a general recommendation for every model, document set, or framework. See the OpenAI retrieval guide for its indexing options.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Small chunks can make search more focused, but may separate a fact from the explanation it depends on. Larger chunks preserve more context but can include unrelated material. Overlap can preserve continuity across boundaries, at the cost of storing and retrieving repeated text. Start with a modest, understandable setup, then inspect the actual passages returned for representative questions before changing chunking.
Build the question-answering workflow
- Accept a question. Keep the original wording; if you rewrite or expand it, retain the user’s intent.
- Retrieve candidate passages. Search the index for chunks related to the question. A vector search is common, but retrieval strategy and filtering can vary by framework and collection.
- Inspect and select evidence. Pass a limited set of relevant chunks onward, including their source labels. More retrieved text is not automatically better if it crowds out useful context.
- Construct a grounded prompt. Clearly distinguish supplied source material from the question, and instruct the model to answer from that material. Ask it to say when the evidence is insufficient rather than inventing missing details.
- Return the answer and evidence. Display or log the retrieved passages and their sources alongside the generated answer so a user can judge whether the response follows from them.
A framework-neutral sketch of the two workflows looks like this:
# Indexing: run when sources are first added or updated
for document in load_documents("docs/"):
chunks = split_into_chunks(document.text, metadata=document.metadata)
vectors = embed([chunk.text for chunk in chunks])
index.store(chunks, vectors)
# Question answering: run for each question
question = receive_question()
question_vector = embed([question])[0]
passages = index.search(question_vector, limit=K)
answer = generate(
prompt="Answer using the supplied passages. If they do not support an answer, say so.",
question=question,
evidence=passages,
)
show(answer, sources=passages)K is a retrieval setting you choose for your system; the sketch intentionally does not prescribe a value. Likewise, real libraries may offer a higher-level chain or query engine that performs several of these steps internally. Keep access to the returned passages even when using that abstraction: they are essential for checking what the model was given.
Evaluate the pipeline with real questions
Evaluation is part of building the workflow, not an optional polish step. LlamaIndex identifies evaluation as a RAG stage and describes using it to check a flow against alternatives and after changes (LlamaIndex’s RAG overview). For a small collection, begin with a short set of representative questions whose answers you can verify from the documents.
PC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware match- Ask a question answered plainly in one passage.
- Ask one that requires combining related passages.
- Ask about information absent from the collection and check that the answer acknowledges the gap.
- Inspect which passages were retrieved, whether their source and wording are relevant, and whether the generated answer is actually supported by them.
- Repeat the same checks after adjusting chunk boundaries, metadata, retrieval settings, or prompts.
Failures point to different parts of the pipeline. If the source never contains the answer, retrieval cannot supply it. If a passage splits a qualification from its claim, revisit chunking or source cleanup. If unrelated passages appear, inspect the query, metadata filters, and retrieval approach. If the evidence is sound but the answer goes beyond it, tighten the prompt or change how unsupported answers are handled. These are checks to perform, not claims of measured performance.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Choose components around your constraints
LangChain and LlamaIndex are implementation options, not prerequisites. You can use a framework to connect loading, indexing, retrieval, and generation, or assemble those components directly. Compare choices by how well their Python interfaces fit your application, how much customization you need, and how much operational work you are prepared to own.
- Local or managed storage: Local storage can suit a small prototype or a deployment where keeping data in your environment is important. A managed service can reduce some infrastructure work but adds a provider and its data-handling terms to the design.
- Data handling: Consider where source text and embeddings are processed and stored, who can access them, and whether your document policies permit sending content to external services.
- Operational responsibility: Account for persistence, backups, updates, access controls, and monitoring rather than treating the vector store as the whole application.
- Customization and integration: Check whether you can control chunking and metadata, inspect retrieved passages, and connect the index to the embedding and generation components you intend to use.
The right stack is the one that meets the collection’s data requirements and gives you enough visibility to inspect retrieval and answer quality. No specific framework or vector store is required by the RAG pattern itself.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
Free tools Windows power users keep installed
One-click scans. No signup required.
