Hugging Face's logo

venetis
/
llama3-8b-hermes-sandals-sample-10k

PEFT
Safetensors
llama
axolotl
Generated from Trainer
Model card Files Community
venetis commited on
Commit
9154f11
•
1 Parent(s): bc835eb

End of training

Browse files
Files changed (2) hide show
  1. README.md +20 -20
  2. adapter_model.bin +1 -1
README.md CHANGED
@@ -29,16 +29,16 @@ strict: false
29
  datasets:
30
  - path: ./data/openhermes2_5_10k.jsonl
31
  type: sharegpt
32
- conversation: llama3
33
  dataset_prepared_path:
34
  val_set_size: 0.15
35
- output_dir: ./outputs_lora-out
36
  hub_model_id: venetis/llama3-8b-hermes-sandals-sample-10k
37
 
38
  data_seed: 117
39
  seed: 117
40
 
41
- chat_template: llama3
42
  adapter: lora
43
  lora_model_dir:
44
  lora_r: 32
@@ -107,7 +107,7 @@ lora_modules_to_save:
107
 
108
  This model is a fine-tuned version of [meta-llama/Meta-Llama-3-8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B) on the None dataset.
109
  It achieves the following results on the evaluation set:
110
- - Loss: 0.8933
111
 
112
  ## Model description
113
 
@@ -141,22 +141,22 @@ The following hyperparameters were used during training:
141
 
142
  | Training Loss | Epoch | Step | Validation Loss |
143
  |:-------------:|:------:|:----:|:---------------:|
144
- | 0.9791 | 0.0102 | 1 | 1.0349 |
145
- | 0.7725 | 0.2538 | 25 | 0.8235 |
146
- | 0.8046 | 0.5076 | 50 | 0.8169 |
147
- | 0.7678 | 0.7614 | 75 | 0.8099 |
148
- | 0.7324 | 1.0152 | 100 | 0.7924 |
149
- | 0.4486 | 1.2487 | 125 | 0.8461 |
150
- | 0.4419 | 1.5025 | 150 | 0.8462 |
151
- | 0.4992 | 1.7563 | 175 | 0.8350 |
152
- | 0.4671 | 2.0102 | 200 | 0.8272 |
153
- | 0.2618 | 2.2411 | 225 | 0.8615 |
154
- | 0.275 | 2.4949 | 250 | 0.8697 |
155
- | 0.2583 | 2.7487 | 275 | 0.8672 |
156
- | 0.3158 | 3.0025 | 300 | 0.8639 |
157
- | 0.2073 | 3.2335 | 325 | 0.8940 |
158
- | 0.1602 | 3.4873 | 350 | 0.8931 |
159
- | 0.1904 | 3.7411 | 375 | 0.8933 |
160
 
161
 
162
  ### Framework versions
 
29
  datasets:
30
  - path: ./data/openhermes2_5_10k.jsonl
31
  type: sharegpt
32
+ conversation: chatml
33
  dataset_prepared_path:
34
  val_set_size: 0.15
35
+ output_dir: ./lora-output-dir
36
  hub_model_id: venetis/llama3-8b-hermes-sandals-sample-10k
37
 
38
  data_seed: 117
39
  seed: 117
40
 
41
+ chat_template: chatml
42
  adapter: lora
43
  lora_model_dir:
44
  lora_r: 32
 
107
 
108
  This model is a fine-tuned version of [meta-llama/Meta-Llama-3-8B](https://huggingface.co/meta-llama/Meta-Llama-3-8B) on the None dataset.
109
  It achieves the following results on the evaluation set:
110
+ - Loss: 0.8913
111
 
112
  ## Model description
113
 
 
141
 
142
  | Training Loss | Epoch | Step | Validation Loss |
143
  |:-------------:|:------:|:----:|:---------------:|
144
+ | 0.9567 | 0.0102 | 1 | 1.0036 |
145
+ | 0.7583 | 0.2545 | 25 | 0.8184 |
146
+ | 0.8226 | 0.5089 | 50 | 0.8238 |
147
+ | 0.7471 | 0.7634 | 75 | 0.8094 |
148
+ | 0.7339 | 1.0178 | 100 | 0.7954 |
149
+ | 0.4737 | 1.2494 | 125 | 0.8393 |
150
+ | 0.4723 | 1.5038 | 150 | 0.8395 |
151
+ | 0.5529 | 1.7583 | 175 | 0.8327 |
152
+ | 0.4288 | 2.0127 | 200 | 0.8277 |
153
+ | 0.2476 | 2.2468 | 225 | 0.8617 |
154
+ | 0.2566 | 2.5013 | 250 | 0.8676 |
155
+ | 0.2787 | 2.7557 | 275 | 0.8654 |
156
+ | 0.3477 | 3.0102 | 300 | 0.8648 |
157
+ | 0.1912 | 3.2392 | 325 | 0.8909 |
158
+ | 0.1868 | 3.4936 | 350 | 0.8912 |
159
+ | 0.1864 | 3.7481 | 375 | 0.8913 |
160
 
161
 
162
  ### Framework versions
adapter_model.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:11818ca0411c345a582973e166a7ea0a185370f4a65a9a42478da155eee042dd
3
  size 4370694462
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0ad75736c96439fef7ac6573457990b3614d98c25164d676e2e68bac32c7806a
3
  size 4370694462