2 docs tagged with "llm-latency"

Debug LLM latency on Kubernetes by separating gateway time, queue wait, prefill, decode, GPU pressure, model readiness, and rollout behavior.

Field note for debugging LLM latency on Kubernetes when pods are healthy but users still wait for time to first token.