qwerrwe / TODO.md
winglian's picture
fdsp config dict fix, todo list, add torchdistx support
ad2b48c
|
raw
history blame
262 Bytes
# todo list
- [] Validation of parameters for combinations that won't work
## things that are known not to work
- FSDP offload and gradient_checkpointing - https://github.com/pytorch/pytorch/issues/82203
- adamw_bnb_8bit doesn't play well with FSDP offload