Lesson 0001

Follow One Request to a Hosted LLM

The first infrastructure skill is being able to point at each layer a request crosses. If you can trace one request, debugging becomes a sequence of small checks instead of a vague feeling that "the server is broken."

The Win

By the end of this lesson, you should be able to explain what happens when a user calls an endpoint like https://llm.example.com/v1/chat/completions, and name the layer most likely responsible when that request fails.

The mental model is deliberately simple: address, route, listener, policy, process, resource. Most hosting problems fit one of those buckets.

The Request Path

Client A browser, app, SDK, curl command, or another backend.
DNS The domain name becomes an IP address.
Internet Route Packets travel toward the public host.
Firewall Provider rules and host rules allow or block the port.
Reverse Proxy Caddy, Nginx, Traefik, or Envoy handles HTTPS and routing.
LLM Server vLLM, TGI, llama.cpp, or another process runs inference.
GPU and Model Drivers, VRAM, weights, context length, and batch settings matter.

The Important Separation

Public edge

DNS, public IP, provider firewall, host firewall, and reverse proxy decide whether outsiders can reach your service.

Private backend

The LLM server should often listen on 127.0.0.1 or a private Docker network, not directly on the public internet.

Runtime capacity

Even when networking is correct, inference can fail from missing model files, insufficient RAM or VRAM, driver mismatch, or overloaded concurrency.

A Minimal Shape

A first serious deployment often looks like this:

client
  -> DNS: llm.example.com
  -> VPS public IP
  -> host firewall allows 443 only
  -> Caddy terminates HTTPS
  -> Caddy reverse proxies to 127.0.0.1:8000
  -> vLLM/TGI/llama.cpp serves the model privately
  -> tokens stream back to the client

This is why --host 0.0.0.0 and --host 127.0.0.1 are not small details. Binding to all interfaces exposes a process broadly if policy allows it. Binding to localhost means only local processes, such as the reverse proxy, can reach it.

Retrieval Practice

Answer from memory. Use short phrases. The feedback is immediate, but the goal is storage strength, not getting it perfect on the first attempt.

Micro-Lab

You do not need a VPS yet. Practice the listener idea locally:

# Terminal 1
python3 -m http.server 8000 --bind 127.0.0.1

# Terminal 2
curl -I http://127.0.0.1:8000

That tiny server is standing in for an LLM server. Later, the same shape becomes vllm serve ... --host 127.0.0.1 --port 8000, with a reverse proxy in front.

What To Ask Me Next

Ask for Lesson 0002 when you want to continue. The next best topic is usually SSH and first VPS hardening: keys, users, updates, UFW, and how not to lock yourself out.

Primary Sources