Hugging Face's logo

Spaces:
Dovakiins
/
qwerrwe
Build error

App Files Community
winglian commited on
Commit
d0d7eaa
•
1 Parent(s): a6ebf57

update openllama and clean up paths

Browse files
Files changed (6) hide show
  1. README.md +8 -8
  2. examples/{gptj-qlora/config.yml → gptj/qlora.yml} +0 -0
  3. examples/openllama-3b/README.md +16 -0
  4. examples/{lora-openllama-3b/config.yml → openllama-3b/lora.yml} +2 -2
  5. examples/{qlora-openllama-3b/config.yml → openllama-3b/qlora.yml} +2 -2
  6. examples/qlora-openllama-3b/README.md +0 -6
README.md CHANGED
@@ -16,14 +16,14 @@
16
 
17
  ## Axolotl supports
18
 
19
- | | fp16/fp32 | fp16/fp32 w/ lora | qlora | gptq | gptq w/ lora | gptq w/flash attention | flash attention | xformers attention |
20
- |----------|:----------|:------------------|-------|------|:-------------|------------------------|-----------------|--------------------|
21
- | llama | ✅ | ✅ | ✅ | ✅ | ❓ | ✅ | ✅ | ✅ |
22
- | Pythia | ✅ | ✅ | ✅ | ❌ | ❓ | ❌ | ❌ | ❓ |
23
- | cerebras | ✅ | ✅ | ✅ | ❌ | ❓ | ❌ | ❌ | ✅ |
24
- | mpt | ✅ | ❌ | ❓ | ❌ | ❓ | ❌ | ❌ | ❓ |
25
- | falcon | ✅ | ✅ | ✅ | ❌ | ❓ | ❌ | ❌ | ✅ |
26
- | gpt-j | ✅ | ✅ | ✅ | ❌ | ❓ | ❌ | ❓ | ✅ |
27
 
28
 
29
  ## Quickstart ⚡
 
16
 
17
  ## Axolotl supports
18
 
19
+ | | fp16/fp32 | lora | qlora | gptq | gptq w/ lora | gptq w/flash attn | flash attn | xformers attn |
20
+ |----------|:----------|:-----|-------|------|:-------------|-------------------|------------|---------------|
21
+ | llama | ✅ | ✅ | ✅ | ✅ | ❓ | ✅ | ✅ | ✅ |
22
+ | Pythia | ✅ | ✅ | ✅ | ❌ | ❓ | ❌ | ❌ | ❓ |
23
+ | cerebras | ✅ | ✅ | ✅ | ❌ | ❓ | ❌ | ❌ | ✅ |
24
+ | mpt | ✅ | ❌ | ❓ | ❌ | ❓ | ❌ | ❌ | ❓ |
25
+ | falcon | ✅ | ✅ | ✅ | ❌ | ❓ | ❌ | ❌ | ✅ |
26
+ | gpt-j | ✅ | ✅ | ✅ | ❌ | ❓ | ❌ | ❓ | ✅ |
27
 
28
 
29
  ## Quickstart ⚡
examples/{gptj-qlora/config.yml → gptj/qlora.yml} RENAMED
File without changes
examples/openllama-3b/README.md ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # openllama-3b
2
+
3
+ Basic full tune
4
+ ```shell
5
+ accelerate launch scripts/finetune.py examples/qlora-openllama-3b/config.yml
6
+ ```
7
+
8
+ LoRA
9
+ ```shell
10
+ accelerate launch scripts/finetune.py examples/qlora-openllama-3b/lora.yml
11
+ ```
12
+
13
+ QLoRA
14
+ ```shell
15
+ accelerate launch scripts/finetune.py examples/qlora-openllama-3b/qlora.yml
16
+ ```
examples/{lora-openllama-3b/config.yml → openllama-3b/lora.yml} RENAMED
@@ -1,5 +1,5 @@
1
- base_model: openlm-research/open_llama_3b_600bt_preview
2
- base_model_config: openlm-research/open_llama_3b_600bt_preview
3
  model_type: LlamaForCausalLM
4
  tokenizer_type: LlamaTokenizer
5
  load_in_8bit: true
 
1
+ base_model: openlm-research/open_llama_3b
2
+ base_model_config: openlm-research/open_llama_3b
3
  model_type: LlamaForCausalLM
4
  tokenizer_type: LlamaTokenizer
5
  load_in_8bit: true
examples/{qlora-openllama-3b/config.yml → openllama-3b/qlora.yml} RENAMED
@@ -1,5 +1,5 @@
1
- base_model: openlm-research/open_llama_3b_600bt_preview
2
- base_model_config: openlm-research/open_llama_3b_600bt_preview
3
  model_type: LlamaForCausalLM
4
  tokenizer_type: LlamaTokenizer
5
  load_in_8bit: false
 
1
+ base_model: openlm-research/open_llama_3b
2
+ base_model_config: openlm-research/open_llama_3b
3
  model_type: LlamaForCausalLM
4
  tokenizer_type: LlamaTokenizer
5
  load_in_8bit: false
examples/qlora-openllama-3b/README.md DELETED
@@ -1,6 +0,0 @@
1
- # qlora-openllama-3b
2
-
3
- ```shell
4
- accelerate launch scripts/finetune.py examples/qlora-openllama-3b/config.yml
5
-
6
- ```