qwerrwe / configs

Commit History

swap batch size for gradient accumulation steps to decouple from num gpu
c2a0792

winglian commited on

Update wandb_log_model on vicuna_13B_4bit_reflect.yml
e0ccacc
unverified

Viktorius Suwandi commited on

Update wandb_log_model on cerebras_1_3B_alpaca.yml
b6a539b
unverified

Viktorius Suwandi commited on

Update wandb_log_model on pythia_1_2B_alpaca.yml
abddcf4
unverified

Viktorius Suwandi commited on

Update wandb_log_model on llama_7B_jeopardy.yml
15aabd2
unverified

Viktorius Suwandi commited on

Update wandb_log_model on llama_65B_alpaca.yml
232b931
unverified

Viktorius Suwandi commited on

Update wandb_log_model on llama_13B_alpaca.yml
0736f4f
unverified

Viktorius Suwandi commited on

Update wandb_log_model on llama_7B_alpaca.yml
d77d736
unverified

Viktorius Suwandi commited on

Update wandb_log_model on galactica_1_3B.yml
2aacf75
unverified

Viktorius Suwandi commited on

Update wandb_log_model on llama_7B_4bit.yml
7187134
unverified

Viktorius Suwandi commited on

Update wandb_log_model on stability_3b.yml
0d14e95
unverified

Viktorius Suwandi commited on

Update wandb_log_model on gpt_neox_20b.yml
84fc217
unverified

Viktorius Suwandi commited on

Update wandb_log_model on quickstart.yml
f317296
unverified

Viktorius Suwandi commited on

Update wandb_log_model on sample.yml
42a971d
unverified

Viktorius Suwandi commited on

refactor(param): rename load_4bit config param by gptq
dd00657

Thytu commited on

fix config for parity with previous change
165da58

winglian commited on

Jeopardy bot! (#17)
a12fb0a
unverified

winglian commited on

update stablelm config
4818380

winglian commited on

fix dataset handling, support galactica
4a17a4c

winglian commited on

tweaks to data loading, 8 bit adam, accelerate and deepspeed
097d367

winglian commited on

fix sharegpt handling from hf, don't worry about loading llama if using earlier transformers release
8d43785

winglian commited on

various bugfixes
94f5e41

winglian commited on

quickstart instructions for starting from runpod (#5)
0a472e1
unverified

winglian commited on

WIP large refactor to make finetune script a little more manageable (#3)
6045345
unverified

winglian commited on

add support for alpaca reflect training (#2)
81de0ef
unverified

winglian commited on

fix lora target module, require explicit flash attention, fix min logging steps, don't use adam8bit for int4, hash prepared datasets, support hf hub datasets
87e073d

winglian commited on

4bit quantized support (wip)
77fca25

winglian commited on

deepspeed doesn't work with flash-attn, and the gpu savings w flash attn are better than the deepspeed headaches
d1aed4c

winglian commited on

add llama 7b config and fiz lora_fan_in_fan_out for llama (copy pasta bug)
d060c80

winglian commited on

more logging, wandb fixes
05fffb5

winglian commited on

improve prepared dataset loading, fix inference
b164725

winglian commited on

helpful info output
937f44f

winglian commited on

various bugfixes
80b2ed2

winglian commited on

more fixes and prep for llama training
949a27b

winglian commited on

config chooser, update readme instructions, device config, llama flash attention, debug out the labels, fix config key checks, other bugfixes
f2a2029

winglian commited on

make it work with pythia in the cloud
8d959a7

winglian commited on

WIP for axolotl trainer
ce24f5e

winglian commited on