Ask HN: MacBook vs. Dedicated GPU for LLM

This Hacker News thread compares using a MacBook (Apple Silicon, with unified memory) versus a dedicated NVIDIA GPU for running large language models (LLMs) locally. - Apple's M-series chips (M1/M2/M3/M4) use "Unified Memory" — the same pool of RAM is shared between CPU and GPU. This means a Mac with 128GB of RAM can load very large models (70B+ parameters) that wouldn't fit on a consumer GPU's VRAM (typically 12-24GB). However, the GPU compute speed is much slower than an NVIDIA card. - Dedicated GPUs (NVIDIA RTX 4090, A6000, etc.) have fast VRAM and highly optimized matrix math (CUDA cores, Tensor Cores), making them much faster for inference and essential for training. The trade-off is limited VRAM capacity. - Tools like llama.cpp, Ollama, and LM Studio run on Mac. Performance depends on model size (parameters) vs. available RAM, and whether the model fits entirely in memory or is "offloaded" in layers. - The key metric: a Mac can run big models slowly; a dedicated GPU runs small models fast. The choice depends on whether you need raw throughput (GPU) or the ability to run large models at all (Mac).