qwerrwe / src

Commit History

optionally define whether to use_fast tokenizer
47d601f

winglian commited on

Support loading data files from a local directory
9bdd30c

utensil commited on

add validation and tests for adamw hyperparam
cb9d3af

winglian commited on

support adamw and grad norm hyperparams
6d0ee4b

winglian commited on

Merge pull request #212 from OpenAccess-AI-Collective/doc-20230615-v1
a81f52d
unverified

winglian commited on

Merge pull request #214 from OpenAccess-AI-Collective/fix-tokenizing-labels
1925eaf
unverified

winglian commited on

add float16 docs and tweak typehints
88e17ff

winglian commited on

bugfix for potential off by one
7925ddc

winglian commited on

style correction
136522f

maciej.karasek commited on

issue #205 bugfix
556fe40

maciej.karasek commited on

update alpaca_chat prompts for instructions to explainn the conversation
4b43a66

winglian commited on

add axolotl trainer and quadratic warmup
7dc580b

winglian commited on

Merge branch 'main' into flash-optimum
fd2c981
unverified

winglian commited on

Merge pull request #187 from OpenAccess-AI-Collective/strip-peft-device-map
93dacba
unverified

winglian commited on

Merge pull request #177 from NanoCode012/fix/landmark-patch
8002ffb
unverified

winglian commited on

Merge pull request #192 from OpenAccess-AI-Collective/sharegpt-custom-prompt
74ef5cc
unverified

winglian commited on

Merge branch 'main' into strip-peft-device-map
5e616d9
unverified

winglian commited on

Merge pull request #159 from AngainorDev/patch-1
8e568bb
unverified

Nanobit commited on

add typehints
c7dee56

winglian commited on

add new sharegpt, refactor prompt so it can be customized later, add exception if no data is processed
aac4b76

winglian commited on

add check for attr
c9a149f

winglian commited on

new validation for mpt w grad checkpoints
14668fa

winglian commited on

Fix strict and Lint
b565ecf

Angainor commited on

match up gradient checkpointing when using lora w config
fe0b768

winglian commited on

Fix set mem_id for inference and refactor
974dc00

Nanobit commited on

Clean up landmark patching
a6190c8

Nanobit commited on

Fix undefined LlamaForCausalLM and del try except
563b6d8

Nanobit commited on

peft no longer needs device_map
cd0a6f6

winglian commited on

Refactor landmark attention patch
919727b

Nanobit commited on

fix formatting
958da70

winglian commited on

Fix missing cfg.
a808bf9
unverified

Angainor Development commited on

Merge pull request #182 from OpenAccess-AI-Collective/fix-llama-ref
0124825
unverified

winglian commited on

address PR feedback
0c6f928

winglian commited on

add streaming dataset support for pretraining datasets
eea2731

winglian commited on

more gpt-neox long ctx fixes
ab5cd28

winglian commited on

fix bettertransformers save, force it to skip after saving correctly in callback
1a82082

winglian commited on

more tweaks to do pre-training with bettertransformers
1210dc8

winglian commited on

experimental expansion of ctx len
488a67d

winglian commited on

add validation/warning for bettertransformers and torch version
71a43f8

winglian commited on

add support for opimum bettertransformers
1edc30c

winglian commited on

fix for local variable 'LlamaForCausalLM' referenced before assignment
14163c1

winglian commited on

Merge branch 'main' into patch-1
79e2a6f
unverified

Angainor Development commited on

add support to extend context with xpos rope
a03a7d7

winglian commited on

fix for max sequence len across different model types
7f09106

winglian commited on

Fix backward compat for peft
aefb2fc

Nanobit commited on

WIP: Rely on cfg.inference
813cfa4
unverified

Angainor Development commited on

Fix grad checkpoint and outputs param
2a801b0

Nanobit commited on

Fix patching via import instead of hijacking
e44c9e0

Nanobit commited on

Feat: Add landmark attention
55b8542

Nanobit commited on