---
license: apache-2.0
library_name: peft
tags:
- axolotl
- generated_from_trainer
base_model: T3Q-LLM/T3Q-LLM-sft1.0-dpo1.0
model-index:
- name: T3Q-LLM-sft1.0-dpo1.0_4300QA
results: []
---
[
](https://github.com/OpenAccess-AI-Collective/axolotl)
See axolotl config
axolotl version: `0.4.0`
```yaml
base_model: T3Q-LLM/T3Q-LLM-sft1.0-dpo1.0
base_model_config: T3Q-LLM/T3Q-LLM-sft1.0-dpo1.0
model_type: AutoModelForCausalLM
tokenizer_type: AutoTokenizer
is_llama_derived_model: true
hub_model_id: T3Q-LLM-sft1.0-dpo1.0_4300QA
load_in_8bit: false
load_in_4bit: true
strict: false
datasets:
# - path: admin_data.csv
- path: superiort/multiplechoice-4300
type: alpaca
# The below are defaults. only set what's needed if you use a different column name.
# system_prompt: ""
# system_format: "{system}"
# field_system: system
# field_instruction: instruction
# field_input: input
# field_output: output
# format: |-
# Human: {instruction} {input}
# Assistant:
# no_input_format: "{instruction} "
# dataset_prepared_path: yanolja_preprocessed_data
dataset_prepared_path: last_run_prepared
val_set_size: 0.2
output_dir: ./T3Q-LLM-sft1.0-dpo1.0_4300QA
adapter: qlora
lora_model_dir:
# device_map: [0,1,3]
sequence_len: 4096
sample_packing: false
lora_r: 32
lora_alpha: 16
lora_dropout: 0.05
lora_target_modules:
lora_target_linear: true
lora_fan_in_fan_out:
wandb_project: axolotl_T3Q_4300
wandb_entity:
wandb_watch:
wandb_run_id: T3Q_mod_4300
wandb_log_model:
gradient_accumulation_steps: 4
micro_batch_size: 2
num_epochs: 10
optimizer: paged_adamw_32bit
lr_scheduler: cosine
learning_rate: 0.0002
train_on_inputs: false
group_by_length: false
bf16: true
fp16: false
tf32: false
gradient_checkpointing: true
early_stopping_patience:
resume_from_checkpoint:
local_rank:
logging_steps: 1
xformers_attention:
flash_attention: true
warmup_steps: 100
eval_steps: 0.01
save_strategy: epoch
save_steps:
debug:
deepspeed:
weight_decay: 0.0
fsdp:
fsdp_config:
special_tokens:
bos_token: ""
eos_token: "<|im_end|>"
unk_token: ""
pad_token: "" # EOS와 PAD가 동일
```
# T3Q-LLM-sft1.0-dpo1.0_4300QA
This model is a fine-tuned version of [T3Q-LLM/T3Q-LLM-sft1.0-dpo1.0](https://huggingface.co/T3Q-LLM/T3Q-LLM-sft1.0-dpo1.0) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 1.2288
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0002
- train_batch_size: 2
- eval_batch_size: 2
- seed: 42
- distributed_type: multi-GPU
- num_devices: 4
- gradient_accumulation_steps: 4
- total_train_batch_size: 32
- total_eval_batch_size: 8
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 100
- num_epochs: 10
### Training results
| Training Loss | Epoch | Step | Validation Loss |
|:-------------:|:------:|:----:|:---------------:|
| 1.2424 | 0.0093 | 1 | 1.0432 |
| 1.0333 | 0.1023 | 11 | 0.9004 |
| 0.8715 | 0.2047 | 22 | 0.7157 |
| 0.7053 | 0.3070 | 33 | 0.6548 |
| 0.6688 | 0.4093 | 44 | 0.6449 |
| 0.6823 | 0.5116 | 55 | 0.6282 |
| 0.5876 | 0.6140 | 66 | 0.6251 |
| 0.6994 | 0.7163 | 77 | 0.6290 |
| 0.6662 | 0.8186 | 88 | 0.6311 |
| 0.6239 | 0.9209 | 99 | 0.6338 |
| 0.5959 | 1.0233 | 110 | 0.6319 |
| 0.6408 | 1.1256 | 121 | 0.6668 |
| 0.595 | 1.2279 | 132 | 0.6221 |
| 0.5476 | 1.3302 | 143 | 0.6295 |
| 0.587 | 1.4326 | 154 | 0.6569 |
| 0.5867 | 1.5349 | 165 | 0.6208 |
| 0.5895 | 1.6372 | 176 | 0.6264 |
| 0.6581 | 1.7395 | 187 | 0.6208 |
| 0.5872 | 1.8419 | 198 | 0.6290 |
| 0.6314 | 1.9442 | 209 | 0.6243 |
| 0.4397 | 2.0465 | 220 | 0.6591 |
| 0.4568 | 2.1488 | 231 | 0.7095 |
| 0.422 | 2.2512 | 242 | 0.6914 |
| 0.453 | 2.3535 | 253 | 0.7001 |
| 0.4678 | 2.4558 | 264 | 0.6896 |
| 0.4335 | 2.5581 | 275 | 0.6776 |
| 0.4796 | 2.6605 | 286 | 0.6829 |
| 0.4637 | 2.7628 | 297 | 0.6742 |
| 0.4532 | 2.8651 | 308 | 0.6828 |
| 0.4348 | 2.9674 | 319 | 0.6836 |
| 0.2787 | 3.0698 | 330 | 0.8085 |
| 0.2336 | 3.1721 | 341 | 0.8380 |
| 0.2341 | 3.2744 | 352 | 0.7998 |
| 0.2393 | 3.3767 | 363 | 0.8041 |
| 0.2826 | 3.4791 | 374 | 0.8040 |
| 0.2505 | 3.5814 | 385 | 0.8099 |
| 0.3057 | 3.6837 | 396 | 0.8103 |
| 0.2789 | 3.7860 | 407 | 0.7964 |
| 0.269 | 3.8884 | 418 | 0.7891 |
| 0.2493 | 3.9907 | 429 | 0.7958 |
| 0.1193 | 4.0930 | 440 | 0.9242 |
| 0.1143 | 4.1953 | 451 | 0.9331 |
| 0.1147 | 4.2977 | 462 | 0.9112 |
| 0.1351 | 4.4 | 473 | 0.9290 |
| 0.0982 | 4.5023 | 484 | 0.9358 |
| 0.1011 | 4.6047 | 495 | 0.9279 |
| 0.09 | 4.7070 | 506 | 0.9289 |
| 0.1063 | 4.8093 | 517 | 0.9392 |
| 0.1038 | 4.9116 | 528 | 0.9267 |
| 0.0361 | 5.0140 | 539 | 0.9412 |
| 0.0371 | 5.1163 | 550 | 1.0589 |
| 0.033 | 5.2186 | 561 | 1.0253 |
| 0.0426 | 5.3209 | 572 | 1.0482 |
| 0.0357 | 5.4233 | 583 | 1.0388 |
| 0.0355 | 5.5256 | 594 | 1.0566 |
| 0.0373 | 5.6279 | 605 | 1.0470 |
| 0.0395 | 5.7302 | 616 | 1.0581 |
| 0.0366 | 5.8326 | 627 | 1.0696 |
| 0.0387 | 5.9349 | 638 | 1.0641 |
| 0.0127 | 6.0372 | 649 | 1.0692 |
| 0.0114 | 6.1395 | 660 | 1.1612 |
| 0.0105 | 6.2419 | 671 | 1.1575 |
| 0.0121 | 6.3442 | 682 | 1.1479 |
| 0.0082 | 6.4465 | 693 | 1.1591 |
| 0.011 | 6.5488 | 704 | 1.1669 |
| 0.0112 | 6.6512 | 715 | 1.1645 |
| 0.0109 | 6.7535 | 726 | 1.1628 |
| 0.0102 | 6.8558 | 737 | 1.1705 |
| 0.0098 | 6.9581 | 748 | 1.1769 |
| 0.006 | 7.0605 | 759 | 1.1840 |
| 0.0064 | 7.1628 | 770 | 1.2016 |
| 0.0063 | 7.2651 | 781 | 1.2133 |
| 0.0058 | 7.3674 | 792 | 1.2182 |
| 0.0056 | 7.4698 | 803 | 1.2218 |
| 0.0057 | 7.5721 | 814 | 1.2234 |
| 0.0059 | 7.6744 | 825 | 1.2245 |
| 0.0057 | 7.7767 | 836 | 1.2247 |
| 0.0048 | 7.8791 | 847 | 1.2247 |
| 0.0054 | 7.9814 | 858 | 1.2246 |
| 0.0051 | 8.0837 | 869 | 1.2252 |
| 0.0059 | 8.1860 | 880 | 1.2261 |
| 0.0053 | 8.2884 | 891 | 1.2272 |
| 0.0057 | 8.3907 | 902 | 1.2275 |
| 0.0056 | 8.4930 | 913 | 1.2280 |
| 0.0052 | 8.5953 | 924 | 1.2283 |
| 0.007 | 8.6977 | 935 | 1.2287 |
| 0.0052 | 8.8 | 946 | 1.2285 |
| 0.005 | 8.9023 | 957 | 1.2289 |
| 0.0056 | 9.0047 | 968 | 1.2288 |
| 0.005 | 9.1070 | 979 | 1.2289 |
| 0.0054 | 9.2093 | 990 | 1.2290 |
| 0.0053 | 9.3116 | 1001 | 1.2288 |
| 0.0049 | 9.4140 | 1012 | 1.2290 |
| 0.0052 | 9.5163 | 1023 | 1.2290 |
| 0.0058 | 9.6186 | 1034 | 1.2291 |
| 0.0059 | 9.7209 | 1045 | 1.2289 |
| 0.0055 | 9.8233 | 1056 | 1.2289 |
| 0.0054 | 9.9256 | 1067 | 1.2288 |
### Framework versions
- PEFT 0.10.0
- Transformers 4.40.1
- Pytorch 2.1.2+cu121
- Datasets 2.15.0
- Tokenizers 0.19.1