23 docs tagged with "kubernetes"

How Kubernetes control plane components shape production architecture decisions.

Field note for GPU node pool Kubernetes incidents where accelerators are expensive, underutilized, or unavailable for LLM inference pods.

Senior guide to GPU node pool design, scheduling, taints, labels, autoscaling, and capacity safety for LLM workloads on Kubernetes.

Debug GPU node pool scheduling for LLM inference with labels, taints, tolerations, GPU requests, quotas, autoscaling buffers, and cost signals.

Decide between KServe and Ray Serve for LLM platforms by ownership model, CRDs, serving graph complexity, autoscaling, rollout, and SRE operability.

Field note for deciding between KServe and Ray Serve for LLM serving on Kubernetes by ownership model, rollback unit, graph complexity, and SRE operability.

Core Kubernetes concepts for senior platform design.

Hands-on Kubernetes LLM labs for vLLM inference, RAG retrieval, observability, and production readiness.

Senior learning map for Kubernetes, platform services, and LLM workloads on Kubernetes.

Debug LLM latency on Kubernetes by separating gateway time, queue wait, prefill, decode, GPU pressure, model readiness, and rollout behavior.

Field note for debugging LLM latency on Kubernetes when pods are healthy but users still wait for time to first token.

Kubernetes networking decisions for service discovery, ingress, egress, and network isolation.

Production Kubernetes practices for scale, security, observability, and recovery.

Reference architecture for a production Kubernetes cluster.

Challenge-style RAG on Kubernetes lab for ingestion, vector retrieval, metadata filters, answer quality, and failure drills.

Production guide for RAG tenant isolation on Kubernetes, covering ingestion metadata, retrieval filters, prompt assembly, evaluation, and unauthorized-document tests.

Field note for RAG on Kubernetes failures where retrieval returns documents outside the user's tenant or authorization boundary.

A senior Kubernetes security hardening checklist for identity, admission, runtime, network, and audit controls.

Kubernetes storage contracts for persistent workloads, backup, restore, and topology.

Challenge-style vLLM Kubernetes lab for GPU scheduling, model cache, OpenAI-compatible serving, probes, metrics, and failure drills.

Production deployment guide for vLLM on Kubernetes covering runtime contract, GPU scheduling, model cache, probes, metrics, and rollout evidence.

Production guide for running vLLM on Kubernetes with GPU scheduling, model cache strategy, runtime flags, probes, metrics, and failure modes.

Workload primitives, placement rules, and scheduling controls for reliable Kubernetes platforms.