Model comparison
FYI. In my opinion, the gemma-2-27b-it-Q4_K_L.gguf model is better than the gemma-2-27b-it-Q5_K_L.gguf
Fascinating... Do you have any examples showing that? Would be very intriguing and I'm not sure why!
FYI. In my opinion, the gemma-2-27b-it-Q4_K_L.gguf model is better than the gemma-2-27b-it-Q5_K_L.gguf
I have tested several times and noticed that some models have this particular thing. The q4 or q5 responds better than q6 and q8. It's about half of them for me. I use LM Studio.
Sorry, I can't provide examples, as they have an nda. But I can say that I have tried these models on the task of code generation and machine translation. In code generation, the code itself has not changed much, but its description has. In machine translation, q5 often had incomprehensible words and incorrectly translated technical terms.