Failing to load oobabooga

#2
by Qnibbles - opened

Hi. Thanks for all the models!
I am using https://github.com/oobabooga/text-generation-webui, main branch commit 91a7370. Latest dependencies pulled from today (although I have tried a few times over the last month), Linux Mint 21.3 Cinnamon. I managed to get bartowski/Liberated-Qwen1.5-7B-exl2 to work but it spits out some weird text sometimes. I have not had much luck with EXL2 generally although its very fast.

I get the following error:

20:36:20-213829 INFO     Loading "Liberated-Qwen1.5-7B-Q6_K.gguf"                                                                                                                  
20:36:20-639466 ERROR    Failed to load the model.                                                                                                                                 
Traceback (most recent call last):
  File "/home/star/IdeaProjects/text-generation-webui/modules/ui_model_menu.py", line 245, in load_model_wrapper
    shared.model, shared.tokenizer = load_model(selected_model, loader)
  File "/home/star/IdeaProjects/text-generation-webui/modules/models.py", line 76, in load_model
    metadata = get_model_metadata(model_name)
  File "/home/star/IdeaProjects/text-generation-webui/modules/models_settings.py", line 68, in get_model_metadata
    bos_token = metadata['tokenizer.ggml.tokens'][metadata['tokenizer.ggml.bos_token_id']]
KeyError: 'tokenizer.ggml.bos_token_id'

Any ideas about how this can be made to work?

Sign up or log in to comment