how much GPU mermory will use?

#4
by praymich - opened
OpenBMB org

Hi, tested the memory consumption of the model loading on Nvidia GPU

  • fp16 17-18G
  • int4 7-8G
Yirany changed discussion status to closed

Sign up or log in to comment