will it local
Pre-RDNA Polaris (gfx803): dropped from modern ROCm — Vulkan/llama.cpp only (HIP route exists on legacy ROCm but is slower/fragile; Vulkan RADV is the community-standard path) China-market 'RX 580 2048SP' variant is a cut-down die (2048 vs 2304 shaders, ~RX 570 class) — verify shader count before buying; a dual-2048SP build is documented on r/LocalLLaMA Vulkan allocator often exposes only ~4GB of the 8GB per card in practice (reported on multi-card Ubuntu setups) — may need per-model layer tuning Ex-mining cards are common; 185W TDP wants a real 8-pin PCIe feed and decent PSU Multi-card works under llama.cpp Vulkan (--device Vulkan0,Vulkan1,...); 4-5 card clusters run 30B-70B quants at 2-5 tok/s (PCIe-limited)
computed band 4.0-10.0 tok/s for 8B Q4_K_M (roofline, 256.0 GB/s); community: 8-15 tok/s Mistral 7B Instruct Q4_K_M (https://www.sitepoint.com/poverty-spec-ai-cluster-rx-580-local-llm); community: 7-13 tok/s Llama 3 8B Q4_K_M (https://www.sitepoint.com/poverty-spec-ai-cluster-rx-580-local-llm); community: ~16 tok/s Qwen3.5-4B Q4_K_M (https://www.reddit.com/r/LocalLLaMA/comments/1rtab9m/rx_580_llamacpp_vulkan_hitting_16_ts_on_qwen354b)
Effective decode window: 0.08–0.21 of 256 GB/s nominal → ~20–54 GB/s effective (llama.cpp decode, Q4_K_M basis; arch window, community-calibrated)
→ 13b-q4 no 11.5 GB vs 8 GB usable — does not fit
→ 14b-q4 offload-partial 9.7 GB vs 8 GB usable — partial CPU offload, expect large speed loss
→ 3b-q4 full 2.8 GB needed of 8 GB usable — headroom for context
→ 4b-q4 full 3.7 GB needed of 8 GB usable — headroom for context
→ 8b-q4 full 6.1 GB needed of 8 GB usable — headroom for context
→ 8-15 tok/s · Mistral 7B Instruct Q4_K_M · source
→ 7-13 tok/s · Llama 3 8B Q4_K_M · source
→ ~16 tok/s · Qwen3.5-4B Q4_K_M · source
Nominal bandwidth: 256.0 GB/s — see the effective decode window above
→ qwen3:0.6b on Raspberry Pi 5 + AI Kit (Hailo-8L) ($110) 0.6B · 0.6GB · needs ~1GB (4k ctx)
→ qwen3:1.7b on Raspberry Pi 5 + AI Kit (Hailo-8L) ($110) 1.7B · 1.4GB · needs ~1.8GB (4k ctx)
→ qwen3:4b on Raspberry Pi 5 + AI Kit (Hailo-8L) ($110) 4B · 2.6GB · needs ~3.2GB (4k ctx)
→ llama3.2:1b on Raspberry Pi 5 + AI Kit (Hailo-8L) ($110) 1B · 1.3GB · needs ~1.4GB (4k ctx)
→ llama3.2:3b on Raspberry Pi 5 + AI Kit (Hailo-8L) ($110) 3B · 2.0GB · needs ~2.4GB (4k ctx)
→ gemma3:1b on Raspberry Pi 5 + AI Kit (Hailo-8L) ($110) 1B · 1.0GB · needs ~1.1GB (4k ctx)
→ phi4:mini on Raspberry Pi 5 + AI Kit (Hailo-8L) ($110) mini 3.8B · 2.5GB · needs ~3GB (4k ctx)
→ moondream:2b on Raspberry Pi 5 + AI Kit (Hailo-8L) ($110) 2B · 1.7GB · needs ~3.7GB (4k ctx)
→ whisper:tiny on Raspberry Pi 5 + AI Kit (Hailo-8L) ($110) tiny · 0.1GB · needs ~2.1GB (4k ctx)
→ whisper on Raspberry Pi 5 + AI Kit (Hailo-8L) ($110) base · 0.3GB · needs ~2.3GB (4k ctx)
→ whisper:small on Raspberry Pi 5 + AI Kit (Hailo-8L) ($110) small · 0.9GB · needs ~2.9GB (4k ctx)
→ whisper:medium on Raspberry Pi 5 + AI Kit (Hailo-8L) ($110) medium · 3.1GB · needs ~5.1GB (4k ctx)
→ kokoro:82m on Raspberry Pi 5 + AI Kit (Hailo-8L) ($110) 82M · 0.3GB · needs ~2.3GB (4k ctx)
→ gemma3:4b on Raspberry Pi 5 + AI Kit (Hailo-8L) ($110) 4B · 3.3GB · needs ~3.8GB (4k ctx)
→ deepseek-r1:7b on Raspberry Pi 5 + AI Kit (Hailo-8L) ($110) distill 7B · 4.7GB · needs ~4.9GB (4k ctx)
→ mistral:7b on Raspberry Pi 5 + AI Kit (Hailo-8L) ($110) 7B · 4.1GB · needs ~4.6GB (4k ctx)
→ llava:7b on Raspberry Pi 5 + AI Kit (Hailo-8L) ($110) 7B · 4.7GB · needs ~6.7GB (4k ctx)
→ qwen3:8b on Tesla P100 (16GB, used) ($135) 8B · 5.2GB · needs ~5.8GB (4k ctx)
→ qwen2.5vl:7b on Tesla P100 (16GB, used) ($135) 7B · 5.6GB · needs ~5.8GB (4k ctx)
→ llama3.1:8b on Tesla P100 (16GB, used) ($135) 8B · 4.9GB · needs ~5.4GB (4k ctx)
→ deepseek-r1:8b on Tesla P100 (16GB, used) ($135) distill 8B · 4.9GB · needs ~5.4GB (4k ctx)
→ mistral-nemo:12b on Tesla P100 (16GB, used) ($135) 12B · 7.1GB · needs ~7.7GB (4k ctx)
→ qwen3:14b 14B — 9.9 GB vs 8 GB usable — partial CPU offload, expect large speed loss
→ gemma3:12b 12B — 9.6 GB vs 8 GB usable — partial CPU offload, expect large speed loss
→ deepseek-r1:14b distill 14B — 9.8 GB vs 8 GB usable — partial CPU offload, expect large speed loss
→ phi4:14b 14B — 9.9 GB vs 8 GB usable — partial CPU offload, expect large speed loss
→ whisper:large-v3 large-v3 — 8.2 of 8 GB usable — barely over; real with q8 KV cache (halves KV) or shorter context
→ sdxl SDXL base — 8.9 GB vs 8 GB usable — partial CPU offload, expect large speed loss
⚠ Pre-RDNA Polaris (gfx803): dropped from modern ROCm — Vulkan/llama.cpp only (HIP route exists on legacy ROCm but is slower/fragile; Vulkan RADV is the community-standard path)
⚠ China-market 'RX 580 2048SP' variant is a cut-down die (2048 vs 2304 shaders, ~RX 570 class) — verify shader count before buying; a dual-2048SP build is documented on r/LocalLLaMA
⚠ Vulkan allocator often exposes only ~4GB of the 8GB per card in practice (reported on multi-card Ubuntu setups) — may need per-model layer tuning
⚠ Ex-mining cards are common; 185W TDP wants a real 8-pin PCIe feed and decent PSU
⚠ Multi-card works under llama.cpp Vulkan (--device Vulkan0,Vulkan1,...); 4-5 card clusters run 30B-70B quants at 2-5 tok/s (PCIe-limited)
$350 — +4GB VRAM +96 GB/s — runs 13B/14B Q4 fully; the classic budget path