OK to upload quantized versions to Ollama library?
It's a lot easier to run models locally using Ollama, both due to the way that tool lets you download and run models and because models in the Ollama library are quantized by default. It's the main tool I recommend others to check out when they want to use LLMs locally.
Do you have any problem with me quantizing the model and uploading it to the Ollama library at https://ollama.com/library? Asking because this model is behind a custom license here on HF.
For reference, the Llama family of models from Meta follow the same pattern: License agreement that has to be accepted on Huggingface and models distributed freely through other platforms in addition. With the license included but not enforced on those other platforms. See Llama 3.1 in Ollama library as an example: https://ollama.com/library/llama3.1/blobs/0ba8f0e314b4
this could be troublesome with some of the licensing they state in the readme:
"4.2 Geographical Restriction
Only organizations having residence or registered main office in the Nordic countries (Denmark, Norway, Sweden, Finland, and Iceland) are permitted to use the Work or Derivative Works thereof without additional consent, provided they comply with the other terms of this License."
As there would be no way to control this by uploading it to ollama. Having a local gguf would likely not be an issue whatsoever, by doing it with llama.cpp on your machine, for example.
Yes that's essentially what I'm asking about here. Does NorwAI have any intention to enforce these criteria? Because IMHO that's pointless and not worth spending resources on. And then open distribution with a LICENSE.md file included should be perfectly fine. They can always go after companies that violate the license later if some company ends up profiting big while in breach. Just like with software licensing in general.
And in the meantime the tinkerers and individual contributors would have an easier time trying out these models since they would be easier to spin up. Which I have to believe is in the best interest of NorwAI.