Lesson 01 · The map before the territory
Before you rent a single GPU, you need a map. This lesson gives you the one mental model that organises the entire open-LLM world — so that every service, price, and buzzword you meet later snaps into a slot you already understand.
Here's the unlock. A closed model like Claude or GPT is a service you can only rent. An open-weight model is a file — a few gigabytes of numbers — that you can download and run anywhere you have a big enough GPU.1 Almost everything "open source LLM" reduces to one question:
(Careful with words: most "open source" LLMs are precisely open-weight — you get the file, not the training data. For your purposes that's fine; just check the license on the model card. Apache-2.0 and MIT models are the safe commercial picks.1)
Running any open model is four choices stacked on top of each other. Learn the stack and you can place any tool you meet.
| Decision | The question | Example answers (June 2026) |
|---|---|---|
| Model | Which file? | Qwen3.5, DeepSeek V4, GLM-5.2, Llama 4, Mistral, gpt-oss2 |
| Engine | What software runs it? | llama.cpp (portable) · vLLM (production)3 |
| Host | Whose computer? | A hosted API · a rented GPU · your own box |
| Access | How does your code call it? | An OpenAI-compatible endpoint — almost always4 |
The Host decision is the one your mission really turns on — it's the whole "OpenRouter vs rent GPUs vs buy a box" question. There are exactly four rungs, from least to most control (and least to most operational burden):
Someone else runs the model; you just call it. Zero ops. OpenRouter (one endpoint → 300+ models), DeepInfra/Novita (cheapest tokens), Groq (fastest), Together/Fireworks (also fine-tune).5
≈ $0.06–$1 per 1M tokens for mid-size open models · OpenRouter adds ~5.5% on credit top-ups, no per-token markup5
You deploy a model/container; the platform spins a GPU up on demand and scales to zero when idle. Modal, RunPod Serverless, HF Inference Endpoints.6 More control than an API, no idle cost.
Billed by the second of active GPU time · HF Endpoints from ~$0.03–0.50/hr-equivalent at the low end7
Rent a whole GPU machine; run anything (vLLM, experiments, agents). You manage the server. RunPod, Vast.ai (marketplace, cheapest), Lambda.8 This is your "cheaper access if I run it myself" path — and the no-strong-computer playground.
RTX 4090 from ~$0.31/hr · H100 ~$1.87–2.99/hr (down 64–75% since 2024)8
A personal GPU/Mac for experimentation, or an on-prem server "as an employee" for a business. You own everything: privacy, no per-call cost, full control — and all the maintenance.9
Consumer box ~$2–4k (runs up to ~32B comfortably) · serious H100 server $25–40k/GPU + power, cooling, ~$750–3k/mo labor910
Where do your favourite names sit? Hugging Face is the library behind all four — where the model files live (2.4M+ of them), plus its own hosting on rungs 1–2.7 Ollama / LM Studio live on rung 4 (and rung 3 once you rent a GPU): the friendly software that runs a model on a machine you control.3
This reframes your mission's "rent GPUs to run open models cheaper" idea: it's true — but conditional. The skill you're really building is knowing where the break-even sits for a given workload, so you can advise a business honestly instead of cargo-culting "self-host = cheaper."
Retrieval beats re-reading. Answer from memory before you peek — the effort is what makes it stick.
1. The clearest difference between an open model and a closed model like Claude is:
2. You want to call an open model from code with zero infrastructure work. Which rung?
3. Roughly when does self-hosting an open model start beating a per-token API?
4. Switching your app from one open-model provider to another usually means:
Read one thing this week: "Best Open-Source LLMs in 2026" on the Hugging Face blog.2 It's the highest-signal current roundup — licenses, sizes, and what each model is actually for. Don't memorise the rankings (they change monthly); read it to feel the shape of the field and the names that keep recurring.
Then lurk in r/LocalLLaMA — the practitioners' watering hole. It's where you'll find honest, same-week reports of what runs well on what hardware, and real cost talk. This is where wisdom lives; the lessons give you the vocabulary to follow along.
Remember: I'm your teacher, not just a document. If any rung, price, or term here is fuzzy, ask me to expand it, draw it differently, or pressure-test it against your real use case.
Sources (verified June 2026; prices and model names drift monthly — re-check before quoting):