Small BLOOM Model for Functional Testing

Description

I've reduced the size bloom to roughly 0.5GB

This repository hosts a significantly smaller version of the BLOOM model, designed primarily for functional testing purposes. It is an ideal choice for scenarios where computational efficiency and quick iterations are necessary, such as in development and testing environments.

Model Details

The original BLOOM model has been scaled down with the following changes:

Number of Layers (n_layer): Reduced to 12 from the original 70 layers.
Hidden Size (hidden_size): Decreased to 512 from the original 14336.
Number of Attention Heads (n_head): Lowered to 8 from the original 112 heads.

Intended Use

This model is suitable for functional testing and development purposes. It is not recommended for tasks that require high accuracy or complex language understanding and generation capabilities.

Disclaimer

Please note that due to the significant reductions in size and complexity, this model does not retain the full capabilities of the original BLOOM model. Expect limitations in accuracy and depth of language understanding.

crafted by Nasser Ali Alzahrani (@nassersala)