Cheat Sheet — Rent a GPU & Serve a Model

Everything you need to spin a cloud GPU up, serve an open model behind an OpenAI-compatible endpoint, call it, and — crucially — stop paying. Keep this open while you work.

The seven steps

#	Do	Why / watch out
1	Add ~$10–20 credit to RunPod (or Vast.ai, cheaper)	You pay per second/hour the pod runs.
2	Deploy a pod: pick a 24GB GPU (RTX 4090 / L40S), image `vllm/vllm-openai:latest`	24GB fits up to ~32B models at quant.
3	Attach a network volume at `/root/.cache/huggingface`	So the model download survives; no re-paying to re-download.
4	Expose HTTP port 8000 in the pod settings	That's where vLLM listens.
5	Set the start command (below) with `--api-key`	Never skip the key — an open endpoint bills strangers' inference to you.
6	Wait for load, then call the endpoint from your code	First call may 502 for a minute while weights load.
7	TERMINATE the pod when done	The #1 way to blow a budget is a pod left running overnight.

The serve command (vLLM)

vllm serve Qwen/Qwen3-8B-Instruct \
  --host 0.0.0.0 \
  --port 8000 \
  --api-key sk-my-secret-123 \
  --max-model-len 16384

Swap the model for Qwen/Qwen3.6-27B-Instruct once the small one works. --host 0.0.0.0 makes it reachable; without it you'll get connection refused.

Call it — same OpenAI client, new base URL

This is the no-lock-in payoff: your existing OpenAI code works, just repointed.

TypeScript (your stack)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://<POD_ID>-8000.proxy.runpod.net/v1",
  apiKey: "sk-my-secret-123",          // the key you set above
});

const res = await client.chat.completions.create({
  model: "Qwen/Qwen3-8B-Instruct",
  messages: [{ role: "user", content: "Say hi in five words." }],
});
console.log(res.choices[0].message.content);

Python / curl

curl https://<POD_ID>-8000.proxy.runpod.net/v1/chat/completions \
  -H "Authorization: Bearer sk-my-secret-123" \
  -H "Content-Type: application/json" \
  -d '{"model":"Qwen/Qwen3-8B-Instruct",
       "messages":[{"role":"user","content":"Say hi in five words."}]}'

GPU ↔ model fit (24GB-class, Q4)

GPU (rent ~$/hr)	Comfortably runs
RTX 4090 24GB (~$0.31–0.69)	up to ~32B (Qwen3.6-27B, Gemma 4 31B)
L40S 48GB (~$0.79–1.1)	up to ~70B at quant, more context
H100 80GB (~$1.87–2.99)	70B comfortably + high concurrency

Cost discipline (for a ~$20–50 budget)

Rules that keep you solvent

Terminate, don't just stop. A running idle pod still bills. Kill it between sessions.
Start small. Prove the loop on an 8B model + cheap GPU before renting an H100.
Use the network volume so you don't re-download weights every session.
Set a spend alert. $0.50/hr × 24h × 7 = $84 — a forgotten pod eats your whole budget in a week.
On-demand while learning, not spot — you don't want your work reclaimed mid-experiment.

Troubleshooting

Symptom	Likely cause
502 Bad Gateway	Server still loading weights, or wrong host/port. Wait, then check binding.
CUDA out of memory	Model too big for the GPU. Smaller model, lower `--max-model-len`, or bigger GPU.
Slow / CPU-bound	`nvidia-smi` shows 0% GPU → wrong image or driver. Use the vLLM image.
401 Unauthorized	Missing/mismatched `--api-key` vs the key in your client.