How much memory a language model needs to run on your own machine: its weights at the precision you pick, the memory its context takes, and room for the runtime, checked against your graphics card, Mac or AI desktop.
Working it out…
More tools
Worked out on your own device; nothing is sent. These are estimates: real use differs by a few percent with the runtime (Ollama, llama.cpp, LM Studio, vLLM), its settings and the file you download, and vLLM claims most of a card by default. GB here are the 1,024³-byte kind that cards and operating systems count. Each model’s shape comes from its own configuration file.