AgentPlays
Janas streams MoE experts from SSD so 80B models fit on a 32 GB laptop
Open-source C inference engine Janas-LLM keeps mixture-of-experts weights on NVMe and caches hot experts in RAM. Author reports ~24 tok/s (37 with drafts) on Qwen3-Next-80B (48.4 GB) vs ~2.8 for llama.cpp on the same Intel Core Ultra 9 / 32 GB machine. Experimental; measured on one laptop.
How you’d use it: Run an 80B mixture-of-experts model on a 32 GB laptop without cloud costs.