Skip to main content
GGenerative AI
    • Start here
    • Learning paths5 paths
    • Modules11 modules
    • Projects5 projects
    • Quizzes
    • Flashcards
    • Homework
    • Interview prep8 question banks
    • Cheat sheets11 sheets
    • Glossary239 terms
    • Code snippets
    • Ask the mentor
    • My notes
    • My progress
SearchStart here
Menu
    • Start here
    • Learning paths5 paths
    • Modules11 modules
    • Projects5 projects
    • Quizzes
    • Flashcards
    • Homework
    • Interview prep8 question banks
    • Cheat sheets11 sheets
    • Glossary239 terms
    • Code snippets
    • Ask the mentor
    • My notes
    • My progress
  1. Reference
  2. Semantic cache

Glossary

Semantic cache

A cache that returns a stored answer for a similar question. It can serve a wrong answer or cross a tenant boundary, so use a strict threshold and scope keys.

Where it is explained

  • Cost and latency optimizationProduction: cost, serving and observability

See also

Terms explained in the same lessons.

  • Batch API
  • Model routing
  • Percentile latency (p50, p95)
  • Streaming
  • Prompt caching
  • Time to first token (TTFT)
Previous termSelf-RAGNext termSemantic memory

All terms, A to Z

GGenerative AI

Learn generative AI from the first token to production systems.

  • Privacy
  • Search