can it run
Comfortably. 24GB of memory against a 7GB working set leaves headroom for context, the OS, and anything else you're running.
weights 4.9GB + KV cache 0.5GB at 4k context vs ~24GB usable → full. 5.4 GB needed of 24 GB usable — headroom for context
computed roofline: 936 GB/s × 0.46–0.76 efficiency window / 4.9GB (Q4_K_M) — bands, never points.
Community check: ~85 tok/s · 8-9B Q4 · 20 runs · inside our computed band — localmaxxing.com
What users report — 5 cited rows:
→ 92 tok/s · Meta Llama 3.1 8B Q4_K_M llama.cpp · source
→ 95.7 tok/s · Meta Llama 3.1 8B Q4_K_M LocalScore median · source
→ 95 tok/s · Qwen 2.5 7B Q4_K_M llama.cpp b3520, 2K ctx · source
→ 108.0 tok/s · Meta Llama 3.1 8B Q4_0 Ollama 0.3.9 (3-4 run avg, runpod sheet) · source
→ 115.3 tok/s · Qwen3 8B Q4_K llama.cpp llama-bench -fa 1, 4K ctx · source
short-context (≤4k) numbers; measured down-scaling at longer contexts: ×0.75 at 16k, ×0.58 at 32k, ×0.40 at 64k, ×0.24 at 128k
→explore the full catalog84 machines indexed · live prices · what each one can run| model file | 4.9GB (GGUF Q4_K_M) |
| minimum memory | 7GB working set |
| this machine | 24GB GDDR6X, 936 GB/s |
| params | 8B |
| license | llama3.1 (gated on HF) |
| price | $1100 |
easy run: ollama pull llama3.1:8b
runs with: ollama · llama.cpp · koboldcpp · lm studio
no alternatives — every machine in the catalog can run this one.
→ rtx-5090 — the next step up from this machine's GPU
NVIDIA (used market) ↗ · Amazon ↗