buun-llama-cpp

A research and development fork of llama.cpp, providing unique KV cache codecs, inference techniques, and bleeding edge features. Why pay 3-bit or 4-bit quality for a context length you only sometimes reach?