Chat with gemma-3-12b-it-qat

This model corresponds to the 12B instruction-tuned version of the Gemma 3 model in GGUF format using Quantization Aware Training (QAT). The GGUF corresponds to Q4_0 quantization. Thanks to QAT, the model is able to preserve similar quality as bfloat16 while significantly reducing the memory requirements to load the model. You can find the half-precision version here.

Links

• https://huggingface.co/google/gemma-3-12b-it
• https://huggingface.co/vinimuchulski/gemma-3-12b-it-qat-q4_0-gguf