lukemarsden
commited on
Commit
•
295b266
1
Parent(s):
77c84e0
Get qlora mistral-7b fine tuning working on a single 4090 (#708)
Browse files
examples/mistral/qlora.yml
CHANGED
|
@@ -43,7 +43,7 @@ wandb_run_id:
|
|
| 43 |
wandb_log_model:
|
| 44 |
|
| 45 |
gradient_accumulation_steps: 4
|
| 46 |
-
micro_batch_size:
|
| 47 |
num_epochs: 1
|
| 48 |
optimizer: adamw_bnb_8bit
|
| 49 |
lr_scheduler: cosine
|
|
|
|
| 43 |
wandb_log_model:
|
| 44 |
|
| 45 |
gradient_accumulation_steps: 4
|
| 46 |
+
micro_batch_size: 2
|
| 47 |
num_epochs: 1
|
| 48 |
optimizer: adamw_bnb_8bit
|
| 49 |
lr_scheduler: cosine
|