prabanta-dev · 2026-10-13 · notable
Janas — runs LLMs bigger than RAM by streaming experts from SSD
Janas is an open-source C engine that runs mixture-of-experts models larger than RAM by reading experts from an NVMe SSD. On a 32 GB laptop it writes Qwen3-Next-80B-A3B at 24 tokens/s, against 2.8 for llama.cpp.
An 80B mixture-of-experts model on a 32 GB laptop, with the weights left on the SSD.
Key specs
| Qwen3 next 80 b write speed | 24 tok/s vs 2.8 (llama.cpp) |
|---|
What is it?
Janas-LLM, the first tool in the Janas family, runs large mixture-of-experts models on ordinary Linux and Windows PCs, with or without a GPU. It is written in C with no dependencies beyond the C library, and its developer reports it is faster than llama.cpp on all 13 models compared in Q4_K_M on one Intel Core Ultra 9 laptop.
How does it work?
The weights stay on disk. For each token the engine reads only the experts it needs from an NVMe SSD and keeps the often-reused ones in an in-memory expert cache, with a --cache limit. An integrated GPU reads prompts through Vulkan, and drafts (speculative decoding) for Qwen and Gemma 4 models raise speed: Qwen3-Next-80B goes from 24 to 37 tokens/s.
Why does it matter?
Large open MoE models usually need more RAM or VRAM than a laptop has. Streaming experts from SSD makes them usable locally, and janas-server adds an OpenAI-style API and editor code completion. The project is experimental: all numbers come from one machine, and the README warns the engine can push the desktop into swap.
Who is it for?
local-LLM users on Linux and Windows laptops
Try it
curl -LO https://github.com/prabanta-dev/janas/releases/latest/download/janas-linux-x86_64.tar.gz