Question bank
Interview Questions — Product Company Level (GenAI System Design)
12 itemsWhat the provenance labels mean
In this bank
Round: GenAI system design (required) · Difficulty: at-level (mid) / stretch (mid) / at-level (senior) Provenance: each prompt is labelled in the table (Common industry pattern or Generated practice). None is verified for a specific company. Treat them as practice, not as reports of what any company asks.
Answer framework (use every time, ~35–45 min)
- Clarify & requirements — users, scale (QPS, corpus size, doc churn), latency/SLO, accuracy bar, cost ceiling, compliance/permissions, failure tolerance. State assumptions out loud.
- Back-of-envelope — tokens/request × QPS → cost/day; embedding + storage size; p95 latency budget split across stages.
- API & data model — endpoints, doc/chunk schema (incl. ACL, version, timestamp).
- Architecture — ingest path, serving path, async paths; name components and why.
- Deep dives — retrieval quality, eval, safety, caching, model routing.
- Failure modes & observability — what breaks, how you detect it, degraded modes.
- Trade-offs — what you chose, what it costs, what you’d change at 10× scale. (Trade-offs and follow-up depth are what usually separate senior answers from mid-level ones. This is a judgment, not data.)
Checklist after each attempt: requirements stated? numbers estimated? retrieval vs generation evaluated separately? permissions addressed? failure modes and monitoring? cost/latency trade-offs named? Note any gaps and practise those next.
| # | Prompt | Key traps interviewers may probe | Provenance | Lesson |
|---|---|---|---|---|
| P1 | RAG Q&A over internal docs (security + eval) | ACLs at retrieval time; eval before launch; freshness | Common industry pattern | Designing a RAG QA system |
| P2 | Research-and-act agent (e.g. book travel) | Irreversible actions, loops, human approval, tracing | Common industry pattern | Designing an agent product |
| P3 | Content moderation/guardrails for a consumer chatbot | Latency vs safety, false positives, serial vs parallel | Generated practice | Designing content moderation and guardrail systems |
| P4 | Support chatbot balancing cost/latency/accuracy at scale | Routing, caching, escalation to humans | Common industry pattern | Cost and latency optimization |
| P5 | Observability + eval infra for millions of queries/day | Sampling, PII, trace storage cost, drift | Generated practice | Observability and eval pipelines |
| P6 | Let an agent take irreversible actions (payments) safely | Confirmation UX, idempotency, limits, audit | Generated practice | Designing an agent product |
| P7 | Model-routing system across small/large models | Router quality, fallback, cost-quality frontier, evals | Common industry pattern | Cost and latency optimization |
| P8 | RAG with strict per-user permissions at scale | Filtering inside ANN, ACL sync lag, cache scoping | Common industry pattern | Designing a RAG QA system |
| P9 | Ingestion for KB updated thousands of times/day | Idempotent upserts, CDC, backfill, freshness SLO | Generated practice | Production deployment of RAG (ingestion section) |
| P10 | Eval framework catching retrieval and generation regressions pre-deploy | Golden set drift, judge bias, CI gates | Common industry pattern | RAG evaluation in depth |
| P11 | Multi-tenant “chat with your data” (text-to-SQL + RAG) | Hallucinated SQL, row-level security, cost | Generated practice | Not covered as a lesson in this course. See scenario S7 in Scenario Based and Designing a RAG QA system for the permissions side |
| P12 | Real-time meeting/call summarisation | Streaming ASR, chunking long input, latency, PII | Generated practice | Cost and latency optimization |
Worked skeleton — P1 (RAG over internal docs)
- Requirements (assumed): 5k employees, 2M chunks, 20 QPS peak, p95 < 4 s to first token (these are invented practice numbers, not benchmarks), answers must cite, per-doc ACLs, nightly + near-real-time updates, abstain when unsure.
- Ingest: connectors (Drive/Confluence/Slack) → parse → structure-aware chunk (+ metadata: source, section, ACL groups, version, updated_at) → embed (batched) → upsert to vector store and BM25 index with deterministic IDs; delete-by-doc on change; DLQ for failures.
- Serve: auth → query rewrite → hybrid retrieve with ACL filter inside the query → rerank top-50→8 → prompt with token budget → generate with citations → citation/groundedness check → stream (perceived latency only; total time is unchanged) → log trace.
- Eval: golden set (query, gold chunk IDs, reference answer); retrieval recall@k/MRR offline; generation faithfulness with gold context; end-to-end; CI gate; weekly sample of prod traffic → labelled.
- Safety: prompt-injection hygiene for retrieved text, PII redaction in logs, no cross-tenant caches.
- Cost/latency: prompt caching of system prefix, small-model router for FAQs, semantic cache scoped by ACL, cap k.
- Failure modes: stale index (freshness monitor), ACL sync lag (fail closed), retriever outage (degrade to keyword), LLM outage (fallback model / retrieval-only snippets).
- 10× scale: shard vector index, move to async ingestion queue, tiered storage, per-tenant namespaces.
Follow-up chain (practise answering without notes)
Provenance: Generated practice.
- “A VP says the bot gave a wrong answer yesterday — how do you debug?” → trace IDs, retrieval inspection.
- “Legal wants deletion within 24h of a doc removal.” → delete propagation, tombstones, cache purge.
- “Cost is 3× budget.” → token accounting by stage, routing, caching.
- “Your golden set has 40 queries. Ship?” → CIs, slice coverage, more labels.