Strata 0.1.41 introduces significant improvements, including an 87% increase in multi-GPU throughput on a four-card R9700 and reduced latency for short prompts on single NVIDIA cards through CPU-assisted prefill. The update eliminates connection resets during high client loads and enhances performance for large models on Windows, with the tokenizer cache speeding up processing times. It also features various fixes, such as addressing deadlocks and improving the handling of malformed API requests, while a number of new options are available but off by default due to potential changes in output. Overall, the release focuses on optimizing multi-GPU configurations and streamlining prompt processing, making it a substantial update for users with compatible hardware

Strata 0.1.41: Multi-GPU Speeds Jump 87% on Radeon R9700

Strata 0.1.41 brings two headline defaults: per-GPU request concurrency that lifts throughput up to 87% on a four-card R9700, and CPU-assisted prefill that trims short-prompt latency on single NVIDIA cards. The multi-GPU fix cuts 4K-to-32K latency roughly in half and erases the connection resets that plagued bursts of 30 to 40 clients. Other gains include a Windows speedup for oversize models, a faster tokenizer cache, and a notable Intel Arc A750 decode bump. It ships on the MIT license with most new flags held off by default until they prove out.

Strata 0.1.41: Multi-GPU Speeds Jump 87% on Radeon R9700 @ Linux Compatible