Recommended Free Tools
iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
For a new conversational RAG application in 2026, Microsoft recommends starting with Azure AI Search agentic retrieval. It can break a complex question into focused searches and merge the results with source references. Choose classic RAG instead when you want a simpler single-query flow, need generally available features only, or already manage orchestration in your application. In either design, the core pipeline is the same: prepare and index content, retrieve relevant evidence, then pass that evidence to a model to compose a grounded response.
Choose the retrieval pattern that fits your application
Classic RAG and agentic retrieval both use Azure AI Search to find evidence for a model. Their main difference is who handles query planning and orchestration. Microsoft describes the trade-offs in its RAG and generative AI overview and agentic retrieval overview.
| Decision | Classic RAG | Agentic retrieval |
|---|---|---|
| How queries run | The application sends a search query; application code handles model handoff and orchestration. | A knowledge base can plan several focused subqueries and run them concurrently. |
| Building blocks | Search index and application-managed orchestration. | Search index, knowledge source or sources, and a knowledge base. |
| Best fit | Simple retrieval, GA-only requirements, an existing orchestration layer, or a preference for a direct single-query path. | Conversational or complex questions and applications that benefit from planned, structured grounding results. |
| Usage considerations | No LLM query-planning call; semantic ranking may still add usage and cost. | Planning and optional answer synthesis consume model tokens; retrieval and semantic ranking also have Azure AI Search billing. |
| Feature maturity | A well-established RAG pattern. | Base capabilities are available in GA, while advanced capabilities depend on preview APIs and schemas. |
There is no universal accuracy or latency winner: test the patterns with representative content and questions. Agentic planning can add work and model-token use; a simpler classic query may be preferable when questions are predictable or the application already controls the retrieval steps.
Build the pipeline in five stages
1. Prepare source content and an index
Choose the documents or other supported source data your application needs, and decide how changes will be refreshed. Design the index around retrieval and response construction, not merely storage. At minimum, plan for stable identifiers, document and source metadata, readable text for each searchable unit, and vector fields when using vector search.
For larger documents, split content into chunks before creating embeddings. Microsoft notes: “Data chunking isn’t a hard requirement, but unless your raw documents are small, chunking is necessary for meeting the token input requirements of embedding models.” See the integrated vectorization overview for supported ingestion patterns.
Do not store only embeddings. Vector conversion is one-way: Azure AI Search cannot turn a vector back into readable source text for a result. Retain the text and metadata alongside each vector so the application can provide useful evidence to the model and identify its origin.
2. Generate embeddings and keep them in sync
With indexer-driven integrated vectorization, configure an index, indexer, and skillset. The skillset can split documents using a suitable strategy—such as Text Split, Azure Content Understanding, or document parsing modes—and then generate embeddings. Documented embedding options include an Azure OpenAI embedding skill, a custom skill, and other model integrations.
Integrated vectorization can also handle vectorization at query time. Whichever route you choose, make sure the query embedding and indexed document embeddings are compatible, and include a refresh process so that changed source content does not leave stale chunks in the index.
3. Retrieve evidence
For a classic baseline, use hybrid retrieval: run keyword and vector search together. Keyword matching helps preserve exact terms such as product names or identifiers; vector search can find semantically similar content even when wording differs. Azure AI Search fuses the result lists, and semantic ranking can rerank a text or hybrid result set. Tune query parameters and, where appropriate, scoring profiles against your own evaluation set rather than assuming one configuration is optimal.
With agentic retrieval, create knowledge source definitions and a knowledge base. The knowledge base can decompose a conversational or multi-part question into focused subqueries, execute them in parallel, apply semantic reranking, and merge results with references. Its planning behavior depends on API version and reasoning configuration; the minimal-reasoning GA path does not offer the same LLM-based planning as the preview path.
4. Send retrieved content to a response model
Retrieval and answer writing are separate jobs. Azure AI Search supplies grounding content and can return source references and execution activity; your application or agent passes the useful content to an LLM and asks it to answer from that evidence. Preserve the references so your interface can show where a response came from.
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Answer synthesis inside the agentic retrieval flow is optional and version-sensitive. The API’s preview path includes it, but a production team can instead retrieve grounding data and have its own application or agent generate the answer. The official agentic retrieval tutorial demonstrates a knowledge base connected to Microsoft Foundry, while the quickstart shows a conversational flow using deployed embedding and language models.
5. Secure the connection
Prefer Microsoft Entra ID role-based access and managed identities where they fit your deployment. In the documented quickstart, the Search service uses a system-assigned managed identity and relevant Search roles; its identity also receives Cognitive Services User for the Foundry resource. That quickstart’s Basic-tier prerequisite applies to its specific configuration, not every possible Azure AI Search design. Key-based authentication is documented as an alternative when role assignment is infeasible.
For a network-isolated Foundry resource with public network access disabled, Microsoft cautions that building agents requires the newer Foundry portal, SDK, or CLI; the classic portal does not support that scenario. Check the tutorial prerequisites and network guidance against your architecture.
Choose the API version before building
Microsoft’s documentation checked October 7, 2026, distinguishes these API paths. Pin the version deliberately: an object created in a preview portal experience can use preview schemas and may need migration before use with the GA REST API.
Quick wins for a faster PC:
Scan for outdated or missing drivers - takes under a minuteDriver Scan →Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →| REST API version | Documented capability and use |
|---|---|
2026-04-01 |
Production use of generally available knowledge source types with minimal, extractive retrieval. |
2026-08-01-preview |
Preview knowledge source types and advanced capabilities, including LLM-based query planning, answer synthesis, non-minimal retrieval reasoning effort, and multi-turn messages. |
Portal experiences expose preview-only agentic retrieval capabilities. Before deploying, verify the current agentic retrieval API guidance and the schemas supported by the version your application will call.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Estimate capacity, latency, and cost for your workload
Tier limits and billing depend on the service configuration. Microsoft’s service limits page, checked in 2026, lists up to three knowledge sources per service on Free and up to 50 on S1. Limits vary by tier, and the page notes exceptions for older Basic and S3 HD services.
The same limits page lists a 10-second minimum, 90-second default, and 600-second maximum for the Azure AI Search retrieve request runtime. These are service request limits, not a promise about end-to-end chatbot response time; planning, model generation, network conditions, and application work can add time.
Agentic retrieval has a free monthly token allowance. Standard billing is pay-as-you-go after that allowance and requires Basic tier or higher. Beginning with REST API 2026-04-01, billing consent for agentic retrieval is separate from semantic ranker. Depending on configuration, costs can include Azure AI Search retrieval-token and semantic-ranking charges, plus model-token charges for query planning or answer synthesis. Consult the current billing consent guidance and your region’s pricing before estimating spend; fixed dollar figures are not meaningful without tier, region, and usage assumptions.
The Serverless Developer pricing model is a preview, is region-limited, has no SLA, and may change in features, limits, or pricing before GA. Microsoft states billing for that preview began September 13, 2026, and that migration to or from other tiers is unsupported. Confirm the current conditions in the service limits and SKU documentation before choosing it.
Evaluate retrieval before relying on generated answers
Build a small test set from real user questions, including exact-name lookups, questions that need multiple documents, ambiguous requests, and questions your content cannot answer. Inspect the retrieved chunks and references separately from the final response. This helps distinguish retrieval failures from generation failures: a fluent answer cannot compensate for missing or irrelevant evidence.
Quick Recap
- Check whether hybrid retrieval returns the expected chunk for exact terms as well as semantically phrased questions.
- Review whether chunk boundaries preserve enough context for a model to interpret each passage.
- For agentic retrieval, inspect planned subqueries, merged evidence, source references, and execution activity.
- Test questions with no supporting content and require the application to say when evidence is insufficient instead of inventing an answer.
- Measure response time and model-token use across the complete application flow, not just the Search retrieve request.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
