Commits · Dovakiins/qwerrwe

fix typo

2eda9e0

tmm1 commited on Aug 3, 2023

scope flash-attn+qlora fix correctly, scope to llama, add comment

78b9efb

tmm1 commited on Aug 3, 2023

move flash-attn monkey patch alongside the others

312a9fa

tmm1 commited on Aug 3, 2023

python 3.10 and 3.11 both work fine, as does pytorch 2.1.0.dev

58d6659

tmm1 commited on Aug 3, 2023

there is no configs folder

cc7e800

tmm1 commited on Aug 3, 2023

feat/llama-2 examples (#319)

dc71d88
unverified

mhenrichsen Mads Henrichsen commited on Aug 3, 2023

ensure flash-attn fixes happen in both adapter/lora modes, and use torch_dtype

248bf90

tmm1 commited on Aug 2, 2023

qlora w flash attention fixes (#333)

77085ea
unverified

winglian commited on Aug 2, 2023

add peft install back since it doesn't get installed by setup.py (#331)

db2a358
unverified

winglian commited on Jul 31, 2023

pin accelerate so it works with llama2 (#330)

6c9a87c
unverified

winglian commited on Jul 31, 2023

fix FSDP save of final model (#329)

894cba0
unverified

winglian commited on Jul 31, 2023

update README for updated docker images (#328)

41a4d15
unverified

winglian commited on Jul 28, 2023

Prune cuda117 (#327)

2c37bf6
unverified

winglian commited on Jul 26, 2023

latest HEAD of accelerate causes 0 loss immediately w FSDP (#321)

9f69c4d
unverified

winglian commited on Jul 24, 2023

update prompts for open orca to match the paper (#317)

3d4984b
unverified

winglian commited on Jul 22, 2023

disable gh cache for first step of docker builds too

ff7f18d

winglian commited on Jul 22, 2023

add runpod envs to .bashrc, fix bnb env (#316)

cf62cfd
unverified

winglian commited on Jul 22, 2023

don't use the gha cache w docker

c5df969

winglian commited on Jul 22, 2023

Merge pull request #307 from OpenAccess-AI-Collective/xgen-user-sharegpt-tokens

40a53ff
unverified

winglian commited on Jul 22, 2023

Merge pull request #306 from ethanhs/xgen

dcdec44
unverified

winglian commited on Jul 22, 2023

Merge pull request #313 from OpenAccess-AI-Collective/tokenizer-llama2-embeddings

3ffb018
unverified

winglian commited on Jul 22, 2023

Merge pull request #299 from OpenAccess-AI-Collective/flash-attention-2

a94f2ee
unverified

winglian commited on Jul 22, 2023

don't resize embeddings to multiples of 32x by default

1066751

winglian commited on Jul 22, 2023

Merge pull request #308 from OpenAccess-AI-Collective/apache2-license

1b63bf1
unverified

winglian commited on Jul 21, 2023

add apache 2.0 license

5cce2a4

winglian commited on Jul 21, 2023

better handling since xgen tokenizer breaks with convert_tokens_to_ids

2a428e8

winglian commited on Jul 21, 2023

pin flash attention 2 to the fix for backwards pass

cdf85fd

winglian commited on Jul 21, 2023

flash attention 2

9b790d3

winglian commited on Jul 20, 2023

Add XGen info to README and example config

3881143

ethanhs commited on Jul 21, 2023

Merge pull request #304 from OpenAccess-AI-Collective/NanoCode012-patch-1

06c61d6
unverified

Nanobit commited on Jul 21, 2023

Merge pull request #300 from OpenAccess-AI-Collective/pytorch-201

262dc29
unverified

winglian commited on Jul 21, 2023

Fix(readme): Improve wording for push model

165907f
unverified

Nanobit commited on Jul 21, 2023

fix sdp attention to use the flash/mem-efficient context manaager

a032c9f

winglian commited on Jul 20, 2023

explicitly pin flash attention 1 to v1.0.9

b06d3e3

winglian commited on Jul 20, 2023

use pytorch 2.0.1

c58034d

winglian commited on Jul 20, 2023

Merge pull request #293 from NanoCode012/fix/tokenize-speed

28fd429
unverified

Nanobit commited on Jul 19, 2023