Feature Request: Support for TurboQuant+ KV Cache Compression (llama.cpp)
Hi LM Studio team! I wanted to put TurboQuant+ on your radar for a future update. It's a recent implementation of the TurboQuant paper (ICLR 2026) that adds extreme KV cache compression (turbo3 and turbo4) to llama.cpp. It allows for massive context windows on Apple Silicon and consumer GPUs (e.g., running a 104B model at 128K context on a Mac Studio) with almost zero speed or quality loss by using attention-gated decoding (Sparse V). The author has a working fork of llama.cpp here: https://github.com/TheTom/turboquant_plus and is preparing an upstream PR. It would be amazing if LM Studio could support the --cache-type-k turbo3 and --cache-type-v turbo3 flags in the UI once this gets merged into upstream llama.cpp, or perhaps as an experimental engine toggle before then.
Posted by ncvinegarbase on Mar 31, 2026, 11:17 PM
Tags
Related issues
- This is the biggest problem with contracts, there's simply no way to actually say "please always execute this check".
- I don't disagree with you on this, I worked my whole life in corporate and haven't worked as a OSS maintainer before, though I will _and ...
- the new VSCode UI, looks cute, i don't like it.
- I've been working on my own private harness for the past 8 months, and I've been collecting ideas from such repos I've stumbled upon.
- The way AI is able to interact with outside resources is pretty impressive, but the quality of code it produces to me is still questionab...
- Add PR-style review comments in Git Diff UI that feed into the active agent thread
Raise real-world problems, find people facing the same issue, and discuss solutions.2,000 issues0 comments
ux (Software)
This is the biggest problem with contracts, there's simply no way to actually say "please always execute this check".
I don't disagree with you on this, I worked my whole life in corporate and haven't worked as a OSS maintainer before, though I will _and ...
the new VSCode UI, looks cute, i don't like it.
I've been working on my own private harness for the past 8 months, and I've been collecting ideas from such repos I've stumbled upon.
The way AI is able to interact with outside resources is pretty impressive, but the quality of code it produces to me is still questionab...
Add PR-style review comments in Git Diff UI that feed into the active agent thread
Thanks for indulging me so far, by the way, I really appreciate this discussion, it's very stimulating.
Migrate GDScript to the GDExtension mechanism and overhaul the implementation
Feature Request: Support for TurboQuant+ KV Cache Compression (llama.cpp)
Hi LM Studio team! I wanted to put TurboQuant+ on your radar for a future update. It's a recent implementation of the TurboQuant paper (ICLR 2026) that adds extreme KV cache compression (turbo3 and turbo4) to llama.cpp. It allows for massive context windows on Apple Silicon and consumer GPUs (e.g., running a 104B model at 128K context on a Mac Studio) with almost zero speed or quality loss by using attention-gated decoding (Sparse V). The author has a working fork of llama.cpp here: https://github.com/TheTom/turboquant_plus and is preparing an upstream PR. It would be amazing if LM Studio could support the --cache-type-k turbo3 and --cache-type-v turbo3 flags in the UI once this gets merged into upstream llama.cpp, or perhaps as an experimental engine toggle before then.
Comments (0)
No comments yet. Start the discussion.