Module 9 of 11
Production: cost, serving and observability
- Production
- 5 lessons
- 2 h 30 min
Chapter 9 · The Lowtide Sleeper
Between one and three in the morning the Sleeper climbs the Mountain Pass in a storm, with a full train and a timetable that changed at midnight. Tobias announces a connection that no longer exists to 400 passengers. When you look for the early warning signs, you find that some were hidden by fuel cuts nobody wrote down, and that nobody was watching the numbers. In this carriage you work with Marta to find what the night cost, why the queue grew, what a dashboard would have shown, and how the Records Car should have heard about midnight.
What you will be able to do
- Measure first, then reduce cost and latency, and state each lever's quality cost.
- Deploy and observe a RAG system with an eval gate and rollback.
Lessons
Cost and latency optimizationCompleted
Read first: How LLMs work, Sampling parameters and the model landscape
LLM inference and servingCompleted
Read first: Cost and latency optimization
Observability and eval pipelinesCompleted
Read first: RAG evaluation and hallucination
Production deployment of RAGCompleted
Read first: RAG evaluation in depth, Observability and eval pipelines
RAG security, reliability and releaseCompleted
Read first: Production deployment of RAG
End of chapter 9 · The Lowtide Sleeper
By three the pass was behind us. Marta hung a gauge on the wall, and for once her thermos was full.