AI engineer interviews
RAG interview questions and answers
RAG means retrieval-augmented generation: the system finds the right documents first, then the model writes an answer from them. It is the most common topic in AI engineer interviews. Interviewers rarely ask for a definition. They hand you a RAG system that gives wrong answers and ask you to find out why.
What they test
What interviewers look for.
- You check retrieval first. If the right document never reached the model, no prompt will fix the answer.
- You split the system into steps: loading documents, cutting them into chunks, search, ranking, and writing the answer. Then you test one step at a time.
- You measure. You build a small test set of real questions with known answers, and you score search and answers apart.
- You know the common fixes and what each one costs: keyword search next to vector search, a reranker, smaller or larger chunks, filters on metadata.
- You think about the product: stale documents, one customer seeing another’s data, speed and price per request.
Questions
RAG debugging questions.
Each one starts with a symptom. A strong answer names the likely causes, then says how to prove which one it is.
Your support assistant cites an outdated policy even though the current document is indexed. How would you isolate the cause?
A good answer covers: trace the returned document ID and version; inspect ingestion, deletion and cache invalidation; compare retrieval quality on a labeled freshness test set.
A support assistant finds the right manual but gives incomplete installation steps. The steps span two chunks. How would you diagnose and fix this?
A good answer covers: inspect retrieved chunks against the complete source passage; preserve step boundaries or retrieve neighboring context; measure answer completeness and retrieval noise on multi-step questions.
A RAG system splits documents into overlapping chunks. Sometimes the same sentence appears multiple times in the context window. How would you diagnose the cause and verify a fix?
A good answer covers: Trace chunk boundaries and overlap logic; Compare context assembly output to original text; Test with edge cases where sentences straddle chunk edges.
A user was removed from a sensitive group, but their RAG query still returns documents accessible only to that group for several minutes after removal. How would you isolate the cause?
A good answer covers: Check if retrieval index snapshots group membership at query time; Review cache or delay in group membership propagation; Compare real-time authorization against retrieval-time access checks.
Your RAG system displays answer confidence based on retrieval similarity scores. Explain the flaw in this approach and describe a safer method for estimating answer correctness.
A good answer covers: Distinguish retrieval similarity from factual correctness; Combine retrieval and generation signals for confidence; Validate confidence estimates with labeled answer quality.
A RAG assistant shows one customer a passage belonging to another tenant. How would you contain and fix this?
A good answer covers: enforce tenant authorization before retrieval; partition or filter indexes and caches by tenant; test cross-tenant leakage and audit exposure.
After a retry storm, your RAG index contains duplicate and conflicting chunks for the same document. How would you determine whether duplicate ingestion jobs are creating competing versions, and how would you confirm the root cause?
A good answer covers: Correlate ingestion job logs to document chunk IDs; Check for idempotency in chunk creation logic; Compare timestamps and content of duplicate chunks.
You notice that ingestion throughput drops to zero after a single problematic document is added. How would you diagnose whether a poison document is blocking the queue, and what mechanism would you use to confirm and isolate the cause?
A good answer covers: Monitor queue progress and stuck jobs; Trace processing logs for repeated failures on one document; Test queue handling of errors and retries for single items.
A user queries for "policies allowing at least 20 days of leave," but your RAG system retrieves documents mentioning "2 days" or "12 days" due to embedding similarity. How would you diagnose and address this numeric range handling issue?
A good answer covers: Identifying numeric entity extraction in queries; Range-aware retrieval or filtering logic; Test queries with varying numeric constraints to verify fix.
Bulk upserts of document vectors sometimes make data visible to all tenants, even when marked private. How would you diagnose this exposure in the vector database configuration?
A good answer covers: Check namespace or tenant partitioning in vector DB; Review upsert logic for correct namespace assignment; Test access to upserted vectors from different tenants.
With the same query vectors, distance metric and document set, exact nearest-neighbor search has good labeled recall but your HNSW search misses relevant neighbors. How would you investigate without immediately changing the embedding model?
A good answer covers: use exact search as a controlled reference to isolate approximate search loss; sweep search exploration such as ef while checking index construction and keeping query filters consistent; plot recall versus latency and memory, considering rebuild cost for construction changes.
A search migration replaces raw dot-product ranking with cosine similarity. Several high-norm documents move down despite unchanged vectors. Explain why this can happen and how you would decide which metric is appropriate.
A good answer covers: explain that dot product includes vector magnitude while cosine normalizes it away; check the embedding model's recommended similarity objective and whether vectors were already normalized; compare labeled relevance under both metrics and handle zero or invalid vectors explicitly.
Your RAG system retrieves plausible documents but misses the relevant one for exact product codes. How would you improve it?
A good answer covers: measure retrieval separately from generation; combine lexical and semantic signals; evaluate recall, ranking and latency tradeoffs.
Your RAG system frequently retrieves multiple near-duplicate documents for a single query, reducing answer diversity. How would you diagnose the root cause and validate the impact on answer quality?
A good answer covers: Analyze deduplication logic in indexing and retrieval; Audit similarity thresholds used for near-duplicate detection; Sample candidate sets to measure factual diversity.
An assistant cites ten websites as independent support for a claim, yet a reviewer finds they republished the same announcement. A less similar primary report contradicting the claim was omitted from context. How would you trace the selection failure and correct it without treating every similar passage as redundant?
A good answer covers: Trace document lineage to distinguish independent evidence from syndicated copies; Inspect how near-duplicate clusters consume the context budget and exclude distinct sources; Validate diversity and contradictory-evidence coverage while preserving independently corroborating reports.
A RAG assistant compares two products, but the answer cites evidence for only one, not both. How would you diagnose whether the missing citation is due to retrieval, synthesis, or grounding logic?
A good answer covers: Check if both product documents were retrieved for the query; Review model prompt and answer generation for citation merging errors; Verify citation attribution logic supports multiple sources per answer.
Some queries and documents share a long prefix but differ in a unique suffix (e.g., 'reset password for admin' vs. 'reset password for user'). Your system truncates embeddings to the first N tokens. How would you diagnose and address the loss of distinguishing information?
A good answer covers: Identifying truncation point in embedding pipeline; Measuring impact on queries with distinguishing suffixes; Testing retrieval on suffix-differentiated queries.
A source document used for a RAG answer is retracted after your system has summarized and cited it. How would you diagnose the risk and track which answers need to be updated or retracted?
A good answer covers: tracking provenance of generated answers; detection and flagging of retracted sources; automated audit and update of affected answers.
Questions
RAG system design questions.
Here you design the system. Say your assumptions, sketch the parts, then go deep on the hardest one.
A retrieval cache is keyed only by content hash. Some documents are public, others private. How would you redesign the cache to prevent users from seeing private content via public cache hits?
A good answer covers: Cache keys must include access scope (e.g., public/private); Prevent cache collisions between access levels; Test cache with mixed-access-content scenarios.
Your RAG pipeline uses a candidate generator with a service account, then reranks using the end user's identity. What is one risk of this setup, and how would you mitigate it?
A good answer covers: Risk of retrieving unauthorized candidates; Consistent identity propagation; Filtering candidates by user authorization.
Your RAG assistant often cites a passage that is topically related but does not actually support the generated answer. What concrete design step would you implement to ensure citations are tightly linked to the specific claims made in answers?
A good answer covers: precise span selection for evidence; semantic alignment check between claim and passage; automated or human verification of citation support.
Design a documentation assistant for 200 teams. Policies change every minute, answers need citations, and teams cannot see one another’s documents. Walk through the read and update paths.
A good answer covers: enforce tenant and document authorization throughout retrieval and caching; version ingestion updates deletions and citations consistently; evaluate freshness groundedness and isolation with observable failure paths.
You’re using Reciprocal Rank Fusion (RRF) to combine two ranked retrieval lists: one from semantic search and one from keyword search. Sometimes, the lists have different lengths or incomparable ranks. How would you design RRF to handle this and verify correctness?
A good answer covers: Normalization of rank positions across lists; Handling missing items or ties in fusion; Testing fusion output on mismatched and matched lists.
Some queries are self-contained and need no retrieval, but your RAG system always adds retrieved context, leading to irrelevant or distracting information. What design would you implement to avoid this, and how would you verify it?
A good answer covers: Detect queries that need no retrieval; Bypass retrieval for such queries; Validate that answer quality improves for self-contained queries.
A user asks about warranty exceptions. The correct paragraph is retrieved but appears in the middle of a long context window. The model ignores it. What context construction strategy could address this, and how would you verify improvement?
A good answer covers: Salience-aware context ordering; Windowed or prioritized evidence packing; Measuring model answer recall against key facts.
A RAG-based answer contains a claim not present in any retrieved passage and provides no citation for it. Under constraints requiring all claims to be grounded, what design change would you make to enforce proper citation coverage and verification?
A good answer covers: Require every answer span to be linked to a source passage; Implement answer validation for uncited content; Design fallback handling for unsupported extrapolations.
A documentation assistant must answer both ordinary how-to questions and questions about which services depend on a deprecated API through several hops. Decide whether to add a graph and outline a minimal design.
A good answer covers: separate passage lookup from explicit multi-hop dependency traversal; compare a graph representation with simpler structured dependency queries and retain passage retrieval for explanatory evidence; track edge provenance and freshness and evaluate known dependency paths, missing edges and answer support.
A question asks which of two contract versions changed a renewal condition. Your retriever usually returns one relevant version first, so MRR looks excellent, but comparisons remain unreliable. Design a retrieval evaluation and serving fix.
A good answer covers: define the required evidence set as both relevant versions with appropriate provenance; measure evidence-set coverage or task completion in addition to first-hit ranking; retrieve and verify the needed version pair within the context budget and abstain when one side is absent.
Design search for a product-support assistant. Queries mix exact part numbers and vague descriptions. Some products have nearly identical names. Explain retrieval and how you would verify improvements.
A good answer covers: combine exact lexical matching and semantic retrieval with structured filters; fuse or rerank candidates while preserving exact identifiers; measure recall and downstream answer quality by query type.
Your document chunking parser is updated to a new version. After re-ingestion, many document chunks receive new IDs, breaking citations and bookmarks. Given the need for backward compatibility, how would you design chunk ID assignment and verify it remains stable across parser changes?
A good answer covers: Decouple chunk IDs from parser implementation details; Design deterministic chunk ID generation based on content; Test ID stability across parser versions with regression checks.
An indexing worker may restart partway through processing a large document. What concrete design ensures no data is lost or duplicated, and the document is fully indexed after recovery?
A good answer covers: idempotent processing of document chunks; checkpointing progress within a document; recovery logic on worker restart.
Your assistant retrieves snippets from multiple product versions, some of which are mutually exclusive. How would you adjust context construction to prevent conflicting information, and how would you verify correctness under tight evidence budget constraints?
A good answer covers: Version-aware evidence selection; Conflict detection among source snippets; Verification against product version constraints.
How to answer
A simple way to answer any RAG question.
- Repeat the symptom
Say what is going wrong and for whom. Ask how often it happens and since when.
- Find the failing step
Look at one bad example. Was the right document in the index? Did search return it? Did the model use it?
- Name the fix and its cost
Pick the smallest change that fixes that step. Say what it does to speed and price.
- Prove it
Run the test set before and after. Then watch real traffic so the problem does not come back.
Questions.
What is RAG in an AI engineer interview?
RAG stands for retrieval-augmented generation. The system searches your documents for the parts that fit the question, then gives them to a language model to write the answer. Interviews test whether you can build one and fix it when it goes wrong.
How do I prepare for RAG interview questions?
Build a small RAG system yourself, then break it on purpose: stale documents, bad chunks, questions with exact codes. Practice finding the cause step by step and saying how you would measure the fix.
What is the most common RAG interview mistake?
Jumping to the prompt or the model. Most wrong answers come from search: the right document was missing, old or ranked too low. Check that first.
Can I practice these questions with Orin?
Yes. The AI Engineer mock interview asks questions like these by voice, with follow-ups based on what you say. You also fix real code in a live editor. After it you get a report with quotes from your answers and what to practice next. Your first interview is free.
Practice RAG questions with follow-ups.
An AI engineer mock interview by voice. Find what broke, fix the code, design the system. First interview free.
Start free interview