Strata 0.2.9 is an open-source local AI inference engine by Niko Veit that enables running a 125-billion-parameter model on standard gaming hardware without the need for a cloud account. The update introduces significant speed improvements for AMD Strix Halo processing, enhanced multi-GPU support, and Unsloth's 1-bit DeepSeek quantization, which compresses a 763-billion-parameter model to about 240 GB with minimal quality loss. Strata's design allows for efficient model processing by activating only a few of its many experts at a time, making it feasible to run large models on less powerful hardware. The project has gained significant traction since its release, amassing around 36,700 stars on GitHub, while remaining free and MIT-licensed

Strata 0.2.9 Adds 1-bit DeepSeek, Strix Halo Speedups, and Multi-GPU Fixes

Strata 0.2.9, the open-source local AI inference engine from developer Niko Veit, runs a frontier-class 125-billion-parameter model on ordinary gaming hardware without a cloud account. The update delivers big speedups for AMD Strix Halo prompt processing, parallel NVIDIA requests on Linux, and multi-GPU support that turns cheap AMD cards into one working accelerator. Most striking is Unsloth's 1-bit DeepSeek quantization, which shrinks a 763-billion-parameter model to roughly 240 GB while losing surprisingly little quality. It's MIT-licensed and free, though it carries known quirks like Strix Halo startup hangs and non-repeatable output at temperature zero.

Strata 0.2.9 Adds 1-bit DeepSeek, Strix Halo Speedups, and Multi-GPU Fixes @ Linux Compatible