Open-Weight AI · Lesson 2

Choosing a Model

The leaderboard is a trap. Three concepts decide which model is actually right for you.

Why this lesson — To ship a feature on an open model you first have to pick one — and there are thousands on Hugging Face. Picking by benchmark rank is how people end up with a model they legally can’t ship or physically can’t run. This lesson gives you a decision order that rules out 95% of the catalogue in seconds, so the choice becomes small and clear.

In Lesson 1 you learned an open model is weights + runtime + hardware, and that size × precision decides what fits. Now: of all the models out there, which weights should you grab? Resist the instinct to sort a leaderboard by score. As the field’s own guides put it — “do not choose by leaderboard rank; choose by fit.”The best open model is “the one that fits your license, hardware, budget, and workflow.” [ref] Fit has three gates, in order.

Gate 1 · License — the legal filter (do this first)

“Open weight” means you can download the model. It does not automatically mean you can ship it in a commercial product. That depends on the licence — and this gate is binary, so apply it before you fall in love with anything.

✓
Apache-2.0 / MIT — the safe ground. Use, modify, ship, even sell, with no user-count strings. Most of Qwen and Mistral (Apache-2.0) and DeepSeek (MIT) live here.
!
Community / custom licences — usable but with terms. Llama ships under Meta’s community licence (not OSI-“open”); fine for almost everyone, but products above 700M monthly active users need a separate deal. Gemma has its own use policy. Read before you build.

Rule of thumb for your mission: default to Apache-2.0 / MIT models so licensing never becomes a launch blocker. Note the licence on the model card; it’s always there. [ref]

Gate 2 · Variant — base vs instruct (the one most beginners miss)

Almost every family ships in two flavours, and picking the wrong one feels like the model is “broken.”

Base / pretrained

A raw autocomplete

Trained only to predict the next token over the internet. Give it a question and it may continue with more questions. Not a chatbot. You want this only as a starting point to fine-tune.

Instruct / chat

Taught to follow you

The base model, further trained on instruction→response pairs (and preference tuning). This is what behaves like the API you already know. Default to this for building features.

So the variant choice is a mission choice: building an app right now → -Instruct. Planning to fine-tune on your own data later → you’ll often start from -Base (we’ll see why in the fine-tuning lessons). On the Hub you’ll literally see two repos, e.g. Qwen3-8B-Base and Qwen3-8B-Instruct.

Gate 3 · Shape & size — what your hardware can hold

Now Lesson 1’s equation does the filtering. But one architectural concept changes how you read model sizes today: dense vs Mixture-of-Experts.

Dense

Every parameter fires

All the weights are used for every token. “8B” means 8B working and 8B to fit in memory. Simple to reason about. Most small/mid models (Gemma 4B, Mistral 24B).

Mixture-of-Experts (MoE)

Total ≫ active

Made of many “expert” sub-networks; each token only routes through a few. Llama 4 Scout = 17B active but 109B total. Fast like a small model, but you must still hold all 109B in memory.

That’s the trap MoE sets for the VRAM equation: speed scales with active params, but memory scales with total params. A headline “17B” MoE can need a data-centre GPU to load. Always size on the total.2026 frontier open models are mostly giant MoEs: DeepSeek 671B (37B active), Qwen 3.5 ~397B (17B active), Mistral Large 3 675B MoE. [ref]

The practical size map

ClassRuns onGood for
Small 1–9B (dense)your Mac, a phone, a 16 GB GPUedge, drafts, simple classification, high volume cheap tasks
Mid 12–32B (dense)one 24–48 GB GPUthe workhorse — most real product features land here
Large 70B (dense)1–2 big GPUs (4-bit)hardest reasoning you’ll self-host on a single box
Frontier MoE 100B–670B+multi-GPU / managed cloudtop quality; usually rent it, don’t host it (Lesson 6)

For your first shippable feature, the honest advice is: start in the 7–14B Instruct, Apache-2.0 band (e.g. a Qwen or Mistral instruct model). Big enough to be genuinely useful, small enough to run on a modest rented GPU or even your Mac — so you learn the whole pipeline before paying for big iron.

Put it together — the chooser

Answer these the way you would for a real project; it walks the three gates and names a sensible starting model. The point isn’t the exact model — it’s feeling the decision order.

What should I start with?

Retrieval practice

1. You load a model, ask a question, and it replies with more questions instead of an answer. Most likely cause?

2. A model is advertised as “17B active.” Why might it still not fit your GPU?

3. You’re shipping a paid product. Which gate should you clear first, before anything else?

▸ Primary source for this lesson

Browse a few real model cards on the Hugging Face model catalogue — filter by task and licence, and notice the -Base vs -Instruct split for yourself. Pair it with HF’s open-LLM & licensing overview. Both are in RESOURCES.md.

Ask me anything. Want me to name 2–3 specific models for your actual feature idea and check their licences? Curious why MoE models exist at all, or what “quantized + MoE” does to the memory math? Or whether to start from base or instruct for your eventual fine-tune? Ask in chat. When ready, say next lesson for Lesson 3 — Quantization: how a 70B model squeezes onto a 24 GB card, and what you sacrifice.