qwerrwe / src /axolotl /monkeypatch /mistral_attn_hijack_flash.py

Commit History

adds llama and mistral dropout support (#858)
db8a8af
unverified

winglian commited on

Mistral: Sliding Window Attention with Flash Attention and Sample Packing (#732)
a045db0
unverified

casperhansen winglian commited on

fix for flash attn w mistral w/o sammple packing (#648)
b2edaae
unverified

winglian commited on

Mistral flash attn packing (#646)
b6ab8aa
unverified

winglian commited on