Latency budgets and SLOs

A latency budget is the maximum time allocated to each component in an inference pipeline. If the end to end SLO is 2 seconds and you have 4 components (auth, retrieval, generation, post processing), each gets a budget. Exceeding any budget breaks the SLO.