Spaces:

Dovakiins
/

qwerrwe

Build error

App Files Files Community

qwerrwe / tests /test_tokenizers.py

Commit History

simplify `load_tokenizer`

efb3b2c

tmm1 commited on Aug 13, 2023

optionally define whether to use_fast tokenizer

47d601f

winglian commited on Jun 25, 2023