qwerrwe / tests /test_prompt_tokenizers.py

Commit History

fix mistral prompt assembly (#982)
7bbaac9
unverified

hamel commited on

Fix prompt assembly for llama (#952)
5ada140
unverified

hamel tokestermw commited on

misc sharegpt fixes (#723)
f30afe4
unverified

winglian commited on

use fastchat conversations template (#578)
e7d3e2d
unverified

winglian commited on

better handling and logging of empty sharegpt turns (#603)
a363604
unverified

winglian commited on

Attention mask and position id fixes for packing (#285)
2bb0b78
unverified

winglian commited on

experimental llama 2 chat support (#296)
3392270
unverified

Jan Philipp Harries Jan Philipp Harries commited on

update prompts for open orca to match the paper (#317)
3d4984b
unverified

winglian commited on

Fixed pre-commit problems, fixed small bug in logging_config to handle LOG_LEVEL env var
b1f4f7a

theobjectivedad commited on

add tests and supoort for loader for sys prompt data
3a38271

winglian commited on

fix test name
1ab3bf3

winglian commited on

ingore duplicate code in tests
baed440

winglian commited on

bugfix for potential off by one
7925ddc

winglian commited on

don't worry about duplicate code here
0136f51

winglian commited on

Fix pre-commit for rebased files
b81c97f

Nanobit commited on

fix relative path for fixtures
cfcc549

winglian commited on

fix relative path for fixtures
e65aeed

winglian commited on

add unit test for sharegpt tokenization
e6fdeb0

winglian commited on