Caching strategies

Caching stores previous computation results to avoid redundant inference. In LLM systems, cache hits save both latency and cost. Effective caching requires choosing the right granularity (exact match, semantic similarity, or prefix) and invalidation strategy.