# Open-Source / Open-Weight AI Resources

Curated, high-trust sources. Knowledge for lessons is drawn from here, not from guesses. Annotated so future-you knows when to reach for each.

## Knowledge

### The model itself (choosing, licenses, weights)
- [Hugging Face Hub](https://huggingface.co) — the primary registry of open models, datasets, and the model cards that document them. Use for: finding a model, reading its license, checking sizes/quant variants.
- [Hugging Face: Best Open-Source LLMs (licensing & fit)](https://huggingface.co/blog/daya-shankar/open-source-llms) — current landscape of Llama / Qwen / Mistral / DeepSeek / Gemma with **license** breakdown. Use for: "which model can I legally ship?" Key fact: Apache-2.0 (Qwen, Mistral) and MIT (DeepSeek) are safest commercially; Llama uses Meta's *community* license (not OSI-open; special terms above 700M MAU).

### Hardware & memory (the VRAM equation, quantization)
- [The Math Behind Local LLMs — VRAM Requirements (DEV)](https://dev.to/bytecalculators/the-math-behind-local-llms-how-to-calculate-exact-vram-requirements-before-you-crash-your-gpu-12n5) — the weights formula + KV-cache overhead. Use for: sizing any model to any GPU.
- [LLM Quantization Explained: Q4, Q8, FP16 & VRAM Tradeoffs](https://llmhardware.io/guides/llm-quantization-guide) — bytes-per-weight per quant level and quality tradeoffs. Use for: deciding how aggressively to quantize.
- [Quantization Explained: Run 70B Models on Consumer GPUs (SitePoint)](https://www.sitepoint.com/quantization-explained-consumer-gpu/) — intuition for why 4-bit barely hurts quality. Use for: the "why does this work?" mental model.

### Running & serving (the runtime)
- [vLLM documentation](https://docs.vllm.ai) — *primary source* for production GPU serving (PagedAttention, continuous batching, OpenAI-compatible server). Use for: standing up a real endpoint on a cloud GPU.
- [Ollama](https://ollama.com) — easiest local-first runtime (wraps llama.cpp / MLX). Use for: first hands-on inference on the Mac.
- [llama.cpp](https://github.com/ggml-org/llama.cpp) — the C++ engine + GGUF quant format underneath Ollama/LM Studio. Use for: constrained/odd hardware and understanding what's underneath.
- [Local LLM Hosting: Complete 2025 Guide (DEV)](https://dev.to/rosgluk/local-llm-hosting-complete-2025-guide-ollama-vllm-localai-jan-lm-studio-more-1dcl) — side-by-side of the runtime landscape. Use for: picking dev vs prod tooling.

### Cloud deployment (managed paths)
- [Amazon Bedrock](https://aws.amazon.com/bedrock/) — managed FMs + **Custom Model Import** (bring your own open-weight/fine-tuned model, serverless). Use for: AWS-native serving without managing GPUs.
- [Google Vertex AI Model Garden](https://cloud.google.com/vertex-ai) — broadest open-model selection + LoRA/PEFT tuning. Use for: GCP-native, data-heavy, want HF models served at scale.
- [Azure AI Foundry](https://ai.azure.com) — model catalog + visual fine-tuning studio; strongest for Microsoft-aligned stacks. Use for: Azure-native serving/tuning.

### Fine-tuning
- [Unsloth — Fine-tuning LLMs Guide](https://unsloth.ai/docs/get-started/fine-tuning-llms-guide) — *primary source* for fast, low-memory LoRA/QLoRA (2× faster, ~80% less memory; runs on free Colab). Use for: actually doing a fine-tune. Includes the [LoRA hyperparameters guide](https://unsloth.ai/docs/get-started/fine-tuning-llms-guide/lora-hyperparameters-guide).
- [Axolotl](https://github.com/axolotl-ai-cloud/axolotl) — config-file-driven fine-tuning (LoRA/QLoRA/full/DPO/ORPO). Use for: reproducible, data-focused tuning runs.
- [BentoML — LLM Fine-tuning (Inference Handbook)](https://bentoml.com/llm/getting-started/llm-fine-tuning) — vendor-neutral overview of when/why/how. Use for: the conceptual framing before touching code.

## Wisdom (Communities)
- [r/LocalLLaMA](https://reddit.com/r/LocalLLaMA) — the highest-signal hub for self-hosting open models: hardware sizing, quant benchmarks, "does X fit on Y GPU?", new releases. Use for: real-world gut-checks before buying compute.
- [Hugging Face forums](https://discuss.huggingface.co) — for `transformers` / PEFT / training-loop questions. Use for: debugging a fine-tune.
- [Unsloth Discord](https://discord.gg/unsloth) — direct help on QLoRA runs. Use for: when a training run OOMs or won't converge.

## Gaps
- No vetted source yet on **cost modelling** (price-per-token of self-hosting vs API break-even). Find one before the cloud-serving lesson.
- No hands-on, reproducible **fine-tuning dataset** example tied to the user's eventual product. Define once the product feature is chosen.