Qwen3.8-27B on 16GB VRAM: Optimized Inference Recipe

Model: Qwen3.8-27B GPU: RTX 4070 Ti SUPER 16GB Context: 100K tokens Decode speed: 47-50 tok/s VRAM usage: 15.93GB Weights (13.5GB hybrid GGUF): - Attention: IQ4_XS - FFNs: IQ3_S KV cache (KVarN compressed): - K: kvarn5 - V: kvarn4 - Most recent 1,024 KV tokens at higher precision Speculative decoding: MTP-2 Key change: kvarn5/kvarn5 -> kvarn5/kvarn4 freed ~6% VRAM, increased usable context from ~88K to 100K tokens. Implementation notes: - Uses BeeLlama.cpp (llama.cpp fork), not stock llama.cpp or standard LM Studio - Community-reported performance; actual results vary with hardware, prompt length, workload, and configuration