adept

/

fuyu-8b

like 980

How many images are there in the folder you preprocess before loading the model? What does the jupyter log say? How much system and GPU memory is being used when you run the notebook?

I would recommend you split the cell in two: one to load the model (run this first), and another one to process the images. My bet is you might be running out of system memory.

Thank you for your quick response.

There are only 2 images in the folder. Jupyter redirects me to this page: https://github.com/microsoft/vscode-jupyter/wiki/Kernel-crashes. The script still crashes at 43 seconds.

So, I split the code into two parts, with downloading and processing the image. However, it still crashes during the model download. The code crashes when it reaches my 16GB of RAM, which is 100% utilization.

What should I do?

Do you have a GPU? How much memory does it have? Fuyu requires ~20 GB of RAM to run in half precision, and double as much in full precision. In addition, if you install accelerate (using pip install accelerate) you can load the model directly on GPU instead of using your system memory and move the weights to GPU later. The following snippet uses both techniques to load the model:

from transformers import FuyuProcessor, FuyuForCausalLM
import torch

model_id = "adept/fuyu-8b"
processor = FuyuProcessor.from_pretrained(model_id)
model = FuyuForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map="cuda")

Yes, I have a GPU and it has 8GB of RAM. On my computer, I have 16GB of DDR4, so that's probably why it crashes. Do you think I should run the code on a cloud, like Google Colab for example?
Thank you in advance!

Yes, 8 GB of GPU RAM is not much for these large models.

Hi,
I have same issue here. I am trying to run this model in GPU but it gets out of memory. I have a 3080Ti with 12GiB and my computer has 32GiB.