Juraj Bednar commited on
Commit
b0cf397
1 Parent(s): 5f79b82

More hints on what to do with CUDA Out of memory errors (#925)

Browse files
Files changed (1) hide show
  1. README.md +4 -0
README.md CHANGED
@@ -1022,6 +1022,10 @@ Please reduce any below
1022
  - `gradient_accumulation_steps`
1023
  - `sequence_len`
1024
 
 
 
 
 
1025
  > `failed (exitcode: -9)`
1026
 
1027
  Usually means your system has run out of system memory.
 
1022
  - `gradient_accumulation_steps`
1023
  - `sequence_len`
1024
 
1025
+ If it does not help, try running without deepspeed and without accelerate (replace "accelerate launch" with "python") in the command.
1026
+
1027
+ Using adamw_bnb_8bit might also save you some memory.
1028
+
1029
  > `failed (exitcode: -9)`
1030
 
1031
  Usually means your system has run out of system memory.