fix: bad dtype for full finetune (#504)

log supervised token count (#448)

Debug tokenization output: Add ability to output text only (no tokens), and/or specify num samples to see (#511)

Added advanced DDP args (#515)

Changed Bench Eval to report metrics correctly by split. Added total accuracy and renamed previously used bench_accuracy to bench_average_accuracy. (#512)

drop empty tokenized rows too (#509)

add eval benchmark callback (#441)

support for datasets with multiple names (#480)

Refactor train cfg cli (#499)

use math.ceil instead of round /cc #498

pad_to_worst_case_seq_len boolean, for testing memory limits (#498)

simplify linear layer locator

fsdp requires params be the same type too (#493)

Fix(tokenizer): Make sure to add pad for CodeLlamaTokenizer (#489)

fix condition and add logging

Merge branch 'main' into patch-4

rename var and reformat

Update src/axolotl/utils/models.py

Update src/axolotl/utils/models.py

let transformers handle adamw_bnb_8bit

ignore: address pr review

ignore: linter

fix: finetune model inference needs the dtype fix to work with flash-attn

fix types w lora (#478)

Fix(tokenizer): Fix condition to add pad token (#477)

improve llama pad token handling (#475)

ReLoRA implementation (with quantization) (#322)

workaround so training doesn't hang when packed dataloader batches aren't even (#461)

recast loralayer, norm, lmhead + embed token weights per original qlora (#393)

always drop samples that are too long (#452)

set env var for FSDP layer to wrap (#453)

add missing positional arg (#450)

fix evals (#447)

support user defined prompters, pretokenized datasets in config, local parquet, local arrow files (#348)

disable eval using multipack for now (#437)

fix comma, not a tuple (#436)

use save_strategy from config if available (#434)

set env for FSDP offload params (#433)

standardize attn hijack patches (#381)

Fix(config): Update handling of deepspeed config (#404)

fix eval steps and strategy (#403)

add utils.data.prepare_dataset

use context manager to run things on rank0 before others (#397)

don't use mask expansion for inference (#392)

Feat(config): add max steps (#387)

Added "epoch" evaluation_strategy (#388)

Feat(config): Add hub_strategy (#386)

don't pass rope_scaling kwarg if it's None (#383)

Fix crash when running without CUDA

try to detect accelerate and only use device_map=None in that case (#373)