Why so many different 8b models
#1
by
Assbang
- opened
Can someone explain??? And how are they different to the blokes version guff.
This is really similar to what the bloke did. Each version corresponds to how "compressed" the model is. For example Q3 (needs ~4GB of VRAM) would be more compressed than Q8 (needs ~8.5GB VRAM)
It's really great because you can choose the model that your laptop can handle. For example, my laptop is a 16gb intel Mac 2020, so I tend to stick to the medium-smaller sized ones.