Skip to main content
GGenerative AI
    • Start here
    • Learning paths5 paths
    • Modules11 modules
    • Projects5 projects
    • Quizzes
    • Flashcards
    • Homework
    • Interview prep8 question banks
    • Cheat sheets11 sheets
    • Glossary239 terms
    • Code snippets
    • Ask the mentor
    • My notes
    • My progress
SearchStart here
Menu
    • Start here
    • Learning paths5 paths
    • Modules11 modules
    • Projects5 projects
    • Quizzes
    • Flashcards
    • Homework
    • Interview prep8 question banks
    • Cheat sheets11 sheets
    • Glossary239 terms
    • Code snippets
    • Ask the mentor
    • My notes
    • My progress
  1. Reference
  2. Percentile latency (p50, p95)

Glossary

Percentile latency (p50, p95)

The time under which 50 percent or 95 percent of requests finish. The tail (p95) shows what the slower users experience, which averages hide.

Where it is explained

  • Cost and latency optimizationProduction: cost, serving and observability

See also

Terms explained in the same lessons.

  • Batch API
  • Model routing
  • Semantic cache
  • Streaming
  • Prompt caching
  • Time to first token (TTFT)
Previous termpass^kNext termpgvector

All terms, A to Z

GGenerative AI

Learn generative AI from the first token to production systems.

  • Privacy
  • Search