LLM Hosting Map

A compact map for thinking about any hosted open-source LLM. When something breaks, locate the failure by asking: address, route, listener, policy, process, or resource?

1. Client

Browser, app, curl, SDK, or another server making an HTTP request.

2. DNS

Turns a name like llm.example.com into an IP address.

3. VPS or GPU Host

The public machine reached by the IP. It has CPU, RAM, disk, network, and maybe GPUs.

4. Firewall

Allows or blocks traffic before it reaches the application.

5. Reverse Proxy

Terminates HTTPS, routes requests, adds headers, and forwards to a private backend.

6. LLM Server

vLLM, TGI, llama.cpp, Ollama, or another process serving model inference.

Six Diagnostic Questions

Question	What It Tests	Typical Tool
Does the name resolve?	DNS record and resolver behavior.	`dig`, `nslookup`
Can traffic reach the host?	Routing, public IP, provider firewall, host firewall.	`ping`, `traceroute`, `nc`
Is anything listening?	Whether a process owns the expected port.	`ss -tulpn`, `lsof -i`
Is policy blocking it?	UFW, security group, Docker port publishing, app auth.	`ufw status`, cloud console, app logs
Is the backend healthy?	Container state, model load, GPU memory, disk, API health.	`docker ps`, `docker logs`, `nvidia-smi`
Is the response safe?	TLS, auth, rate limits, logging, prompt/data exposure.	`curl -v`, proxy config, API gateway logs

Keep These Separate

Layer	Means	Failure Smell
Address	The name or IP the client uses.	Wrong DNS record, stale cache, wrong domain.
Route	The path packets take to the machine.	Host unreachable, provider firewall, wrong network.
Listener	The process bound to a port.	Connection refused, process crashed, wrong bind address.
Policy	Rules deciding who may connect and what they may do.	Timeouts, 401/403, blocked port, missing API key.
Process	The app or model server doing the work.	500s, model load failures, bad config.
Resource	CPU, RAM, VRAM, disk, bandwidth, file descriptors.	OOM, CUDA errors, slow tokens, full disk.