"What are the three variables in the serving-stack decision matrix, and in what order do you ask them?"	"(1) Audience — one developer, a small team, or hundreds of concurrent end users? (2) Hardware — NVIDIA CUDA, Apple Silicon, CPU-only, ROCm? (3) Telemetry posture — open network, regulated subnet, or true air-gap? Ask in this order; telemetry is the most underweighted and the one Pillar 7 builds on."	c3::ft20::recall
"Name the canonical runtime for each quadrant of the decision matrix."	"NVIDIA production (open subnet, many users) -> vLLM. Air-gap / regulated (any HW) -> llama.cpp server. Mac fleet (Apple Silicon) -> mlx-lm. Dev / single user (any HW) -> Ollama. The matrix is a routing function, not a ranking."	c3::ft20::recall
"What three engineering choices made vLLM the production standard on NVIDIA?"	"(1) PagedAttention — KV cache in paged blocks (like OS virtual memory), eliminating fragmentation. (2) Continuous batching — insert new requests into running batches at every token step, keeping GPUs saturated. (3) Multi-GPU tensor parallelism — split one model across GPUs via --tensor-parallel-size N."	c3::ft20::recall
"What is PagedAttention and why does it matter?"	"vLLM stores the KV cache in paged blocks (like an OS's virtual memory) rather than a contiguous tensor. This eliminates the fragmentation that wastes 60-80% of KV-cache memory in naive implementations, letting vLLM pack many more sequences into the same VRAM."	c3::ft20::application
"What is continuous batching and what does it buy you over static batching?"	"Naive batching waits for the longest sequence in a batch to finish before starting the next batch. Continuous batching inserts new requests into running batches at every token step, so GPUs stay saturated and tail latency stays bounded. The throughput advantage over static batching is often 5-10x under realistic load."	c3::ft20::application
"What is vLLM's telemetry posture, and why does it matter for sensitive deployments?"	"vLLM is OpenTelemetry-native: it emits traces/metrics via standard OTel env vars to a collector YOU configure. There is no vendor phone-home, no license check, no call-home server. Point exporters at your own collector (127.0.0.1) or leave them unset and traces are no-ops. This is what makes vLLM acceptable behind a strict egress firewall."	c3::ft20::analysis
"Why is llama.cpp server described as the 'air-gap champion'?"	"Three negative virtues: (1) Single statically-linked binary — no Python, no container runtime, no transitive deps. (2) No network code path for telemetry — nothing to disable, no phone-home flag. (3) Maximum hardware breadth — CUDA, Metal, ROCm, Vulkan, CPU in the same code path. Cut the network cable and it keeps serving. Most self-contained option in the matrix."	c3::ft20::analysis
"State the no-telemetry rule of thumb."	"llama.cpp and vLLM genuinely never need to phone home. Ollama needs the network only for `ollama pull`; once models are local, block the network and bind OLLAMA_HOST=127.0.0.1. For a true air-gap, pre-load the models, then sever the network."	c3::ft20::recall
"What is the air-gap recipe (5 steps), generalized to any runtime?"	"(1) Pre-load models on a connected staging machine. (2) Transfer via approved media (USB, one-way diode, sneakernet). (3) SHA256-verify on the isolated side. (4) Serve with llama.cpp server (preferred) or pre-loaded vLLM/Ollama with egress firewalled. (5) Patch on a schedule via the same media — air-gap means isolated, not invulnerable; known CVEs still apply."	c3::ft20::application
"What does Ollama's official privacy policy say, and what are the three caveats engineers miss?"	"Policy: 'We don't see your prompts or data when you run locally.' True for prompt data. Caveats: (1) Chat history stored in PLAINTEXT locally (~/.ollama) — data-at-rest concern on shared/fleet machines. (2) Misconfigured OLLAMA_HOST=0.0.0.0 exposes an unauthenticated API to the network. (3) Oligo found 6 CVEs in 2025 — patched if you keep it updated; an air-gapped Ollama that never patches carries known CVEs."	c3::ft20::analysis
"At roughly how many concurrent users does Ollama collapse, and why?"	"~5 concurrent users. Response times go from ~3s at 1 user to over a minute at 5+ because Ollama serializes requests and duplicates context per request — no PagedAttention, no real continuous batching. vLLM on the same hardware holds the workload at stable latency. This is why Ollama is a dev tool, not a production server."	c3::ft20::analysis
"Name the six Oligo CVEs' categories disclosed in Ollama in 2025."	"Six vulnerabilities including: CVE-2025-1975 (Denial of Service), CVE-2025-51471 (cross-domain token exposure, v0.6.7), CVE-2025-63389 (authentication bypass in API endpoints, v<=0.12.3). Patched in current releases — the lesson is that air-gapped != invulnerable; an unpatched air-gapped Ollama carries known CVEs."	c3::ft20::recall
"Why is OLLAMA_HOST=0.0.0.0 a known attack surface?"	"The default 127.0.0.1 bind is correct (loopback only). Setting 0.0.0.0 to 'make it work from another machine' exposes an unauthenticated API to the network. Public-internet Ollama instances have been scraped repeatedly, and the 2025 Oligo CVEs (including an auth bypass) make raw exposure actively dangerous. Never bind Ollama broadly without an auth layer (reverse proxy) in front."	c3::ft20::analysis
"What is TGI and when do you choose it over vLLM?"	"TGI (text-generation-inference) is HuggingFace's production server: CUDA-first, high concurrency, and the strongest out-of-box observability — Prometheus metrics and Grafana dashboards are first-class, not bolted on. Choose TGI when your org already runs the HF stack and wants mature metrics without wiring up OTel by hand. Its telemetry, like vLLM's, is yours to point at your own collector."	c3::ft20::application
"What is mlx-lm and when is it the right answer?"	"mlx-lm is Apple's model-serving layer on MLX, the native array framework for Apple Silicon unified memory. It is to Macs what vLLM is to NVIDIA — built for the hardware, not ported to it. Right answer when the fleet is Macs (healthcare workstations, defense laptops) and the constraint is privacy by construction: the model runs locally, the data never leaves the device."	c3::ft20::application
"What is the MLX quantization recommendation, and roughly what does 4-bit buy you?"	"4-bit is the sweet spot: ~75% size reduction versus FP16 with minimal measurable quality loss on standard benchmarks for sub-30B instruct/reasoning models. You rarely quantize yourself — pull pre-quantized models from the mlx-community org on HuggingFace (e.g., mlx-community/Qwen2.5-7B-Instruct-4bit). Serve with `python -m mlx_lm.server`."	c3::ft20::application
"What is the federated local serving pattern, and why is it a privacy advantage?"	"For a Mac fleet, you do NOT stand up one central MLX/vLLM server and share it. You run MLX on EACH workstation. There is no central server holding anyone's prompts — the privacy property is structural, not a policy. Trade-off: it is a different operational model than the vLLM-on-a-GPU-box pattern; teams used to centralized serving need to adjust."	c3::ft20::analysis
"Name the four production deployment patterns."	"(1) Bind loopback (127.0.0.1) for local-only — expose the raw API to the network only deliberately, with auth. (2) OpenAI-compatible API for app integration — standardize on /v1/chat/completions so base_url swap is the only prod-vs-dev change. (3) Concurrency limits + queueing for multi-user (vLLM/TGI have it; Ollama doesn't). (4) Observability wired to YOUR OWN collector (OTel for vLLM, Prometheus for TGI)."	c3::ft20::recall
"Why is the OpenAI-compatible API described as the single highest-leverage integration decision in the serving layer?"	"vLLM, llama.cpp server, Ollama, TGI, MLX, and SGLang ALL speak the /v1/chat/completions dialect. Your app calls one endpoint with a configurable base_url; you swap base_url between local Ollama in dev and a vLLM cluster in prod with NO code change. This lets you run different runtimes per environment without retraining ops on different client stacks."	c3::ft20::analysis
"What is SGLang's signature advantage, and when does it beat vLLM?"	"SGLang uses RadixAttention — prefix caching across requests that share prefixes. It beats vLLM on agentic workloads where many requests share a long system prompt (the prefix is computed once and reused). On workloads with all-unique prompts, vLLM and SGLang are close. Use SGLang alongside vLLM when benchmarked faster for your specific traffic shape."	c3::ft20::analysis
"What is TensorRT-LLM and what is its cost?"	"NVIDIA's own highly-optimized runtime — the lowest-latency option on H100/H200 class hardware. The cost is integration complexity: building engines, paged KV cache tuning, stricter model support. Reach for it when squeezing the last millisecond on flagship NVIDIA silicon and you have the engineering bandwidth to maintain it."	c3::ft20::application
"What is localai and how does it differ from Ollama?"	"localai is an Ollama-COMPATIBLE API surface over a BROADER set of backends (diffusers, audio, vision models — not just text LLMs). Community-maintained. Useful when you want Ollama's DX but need a backend Ollama doesn't cover. Treat its telemetry and security posture as 'varies by backend' — audit before deploying in regulated environments."	c3::ft20::recall
"Why is 'it runs locally' NOT equivalent to 'it is private'?"	"Three reasons: (1) Ollama stores chat history in plaintext on disk — data-at-rest concern. (2) vLLM emits OTel traces to whatever endpoint you point it at — if you misconfigure the collector URL, you can exfiltrate your own data to yourself or a third party. (3) A misconfigured bind (OLLAMA_HOST=0.0.0.0) exposes the API to the network. For HIPAA/defense/air-gap, the telemetry posture is a first-class variable — audit before shipping."	c3::ft20::analysis
"For a hospital with 500 Mac workstations, what is the serving architecture, and why?"	"Federated local serving via mlx-lm on EACH workstation — NOT one central vLLM box. Reasons: (1) patient data cannot leave the device (privacy by construction); (2) there is no central server holding everyone's prompts (no compliance target); (3) MLX is the Mac-native runtime (built for unified memory, not ported). 4-bit quantized models from mlx-community, served via the OpenAI-compatible API. Different operational model than centralized serving."	c3::ft20::application
"What is the single most expensive serving mistake teams make, and how do you avoid it?"	"Using Ollama for production multi-user. Ollama collapses at ~5 concurrent users (no PagedAttention, request serialization, per-request context duplication). Avoidance: use Ollama for dev/single-user; the moment you have real concurrency, move to vLLM or TGI. Benchmark under realistic concurrent load before declaring any runtime production-ready — single-request latency tells you nothing."	c3::ft20::analysis