14 docs tagged with "llm"

Field note for GPU node pool Kubernetes incidents where accelerators are expensive, underutilized, or unavailable for LLM inference pods.

Senior guide to GPU node pool design, scheduling, taints, labels, autoscaling, and capacity safety for LLM workloads on Kubernetes.

Debug GPU node pool scheduling for LLM inference with labels, taints, tolerations, GPU requests, quotas, autoscaling buffers, and cost signals.

Benchmark LLM inference on Kubernetes using latency phases, throughput, GPU pressure, and cost per request.

Compare KServe and Ray Serve for LLM serving on Kubernetes by ownership model, CRDs, serving graph complexity, autoscaling, rollout behavior, and team fit.

Hands-on Kubernetes LLM labs for vLLM inference, RAG retrieval, observability, and production readiness.

Senior learning map for Kubernetes, platform services, and LLM workloads on Kubernetes.

Reference architecture for LLM inference on Kubernetes.

Challenge-style observability lab for Kubernetes LLM workloads covering latency, queueing, GPU saturation, traces, logs, and alerts.

Senior guide to Kubernetes LLM infrastructure with GPU node pools, vLLM, KServe, Ray Serve, RAG, benchmarking, and cost controls.

Compare vLLM, KServe, Ray Serve, and Triton for Kubernetes LLM serving, and link to deeper vLLM Kubernetes and KServe vs Ray Serve guides.

Failure modes and evaluation strategy for production RAG systems on Kubernetes.

Production RAG on Kubernetes guide covering ingestion, retrieval, vector databases, serving, evaluation, authorization, observability, and failure modes.

Reference architecture for retrieval augmented generation on Kubernetes.