Upload README.md

README.md

@@ -0,0 +1,144 @@
+---
+license: creativeml-openrail-m
+tags:
+- stable-diffusion
+- stable-diffusion-diffusers
+- text-to-image
+datasets:
+- ChristophSchuhmann/improved_aesthetics_6.25plus
+library_name: diffusers
+pipeline_tag: text-to-image
+extra_gated_prompt: >-
+  This model is open access and available to all, with a CreativeML OpenRAIL-M
+  license further specifying rights and usage.
+
+  The CreativeML OpenRAIL License specifies: 
+
+
+  1. You can't use the model to deliberately produce nor share illegal or
+  harmful outputs or content 
+
+  2. The authors claim no rights on the outputs you generate, you are free to
+  use them and are accountable for their use which must not go against the
+  provisions set in the license
+
+  3. You may re-distribute the weights and use the model commercially and/or as
+  a service. If you do, please be aware you have to include the same use
+  restrictions as the ones in the license and share a copy of the CreativeML
+  OpenRAIL-M to all your users (please read the license entirely and carefully)
+
+  Please read the full license carefully here:
+  https://huggingface.co/spaces/CompVis/stable-diffusion-license
+      
+extra_gated_heading: Please read the LICENSE to access this model
+
+---
+
+
+# BK-SDM-2M Model Card
+
+BK-SDM-{[**Base-2M**](https://huggingface.co/nota-ai/bk-sdm-base-2m), [**Small-2M**](https://huggingface.co/nota-ai/bk-sdm-small-2m), [**Tiny-2M**](https://huggingface.co/nota-ai/bk-sdm-tiny-2m)} are pretrained with **10× more data** (2.3M LAION image-text pairs) compared to our previous release.
+- Block-removed Knowledge-distilled Stable Diffusion Model (BK-SDM) is an architecturally compressed SDM for efficient text-to-image synthesis. 
+- The previous BK-SDM-{[Base](https://huggingface.co/nota-ai/bk-sdm-base), [Small](https://huggingface.co/nota-ai/bk-sdm-small), [Tiny](https://huggingface.co/nota-ai/bk-sdm-tiny)} were obtained via distillation pretraining on 0.22M LAION pairs.
+- Resources for more information: [Paper](https://arxiv.org/abs/2305.15798), [GitHub](https://github.com/Nota-NetsPresso/BK-SDM), [Demo]( https://huggingface.co/spaces/nota-ai/compressed-stable-diffusion).
+
+
+## Examples with 🤗[Diffusers library](https://github.com/huggingface/diffusers).
+
+An inference code with the default PNDM scheduler and 50 denoising steps is as follows.
+
+```python
+import torch
+from diffusers import StableDiffusionPipeline
+
+pipe = StableDiffusionPipeline.from_pretrained("nota-ai/bk-sdm-tiny-2m", torch_dtype=torch.float16)
+pipe = pipe.to("cuda")
+
+prompt = "a black vase holding a bouquet of roses"
+image = pipe(prompt).images[0]  
+    
+image.save("example.png")
+```
+
+
+## Compression Method 
+
+Adhering to the [U-Net architecture](https://huggingface.co/nota-ai/bk-sdm-tiny#u-net-architecture) and [distillation pretraining](https://huggingface.co/nota-ai/bk-sdm-tiny#distillation-pretraining) of BK-SDM, the difference in BK-SDM-2M is a 10× increase in the number of training pairs.
+
+- **Training Data**: 2,256,472 image-text pairs (i.e., 2.3M pairs) from [LAION-Aesthetics V2 6.25+](https://laion.ai/blog/laion-aesthetics/).
+- **Hardware:** A single NVIDIA A100 80GB GPU
+- **Gradient Accumulations**: 4
+- **Batch:** 256 (=4×64)
+- **Optimizer:** AdamW
+- **Learning Rate:** a constant learning rate of 5e-5 for 50K-iteration pretraining
+
+
+## Experimental Results 
+
+The following table shows the zero-shot results on 30K samples from the MS-COCO validation split. After generating 512×512 images with the PNDM scheduler and 25 denoising steps, we downsampled them to 256×256 for evaluating generation scores.
+
+- Our models were drawn at the 50K-th training iteration.
+
+| Model | FID↓ | IS↑ | CLIP Score↑<br>(ViT-g/14) | # Params,<br>U-Net | # Params,<br>Whole SDM |
+|:---:|:---:|:---:|:---:|:---:|:---:|
+| [Stable Diffusion v1.4](https://huggingface.co/CompVis/stable-diffusion-v1-4) | 13.05 | 36.76 | 0.2958 | 0.86B | 1.04B |
+| [BK-SDM-Base](https://huggingface.co/nota-ai/bk-sdm-base) (Ours) | 15.76 | 33.79 | 0.2878 | 0.58B | 0.76B |
+| [BK-SDM-Base-2M](https://huggingface.co/nota-ai/bk-sdm-base-2m) (Ours) | 14.81 | 34.17 | 0.2883 | 0.58B | 0.76B |
+| [BK-SDM-Small](https://huggingface.co/nota-ai/bk-sdm-small) (Ours) | 16.98 | 31.68 | 0.2677 | 0.49B | 0.66B |
+| [BK-SDM-Small-2M](https://huggingface.co/nota-ai/bk-sdm-small-2m) (Ours) | 17.05 | 33.10 | 0.2734 | 0.49B | 0.66B |
+| [BK-SDM-Tiny](https://huggingface.co/nota-ai/bk-sdm-tiny) (Ours) | 17.12 | 30.09 | 0.2653 | 0.33B | 0.50B |
+| [BK-SDM-Tiny-2M](https://huggingface.co/nota-ai/bk-sdm-tiny-2m) (Ours) | 17.53 | 31.32 | 0.2690 | 0.33B | 0.50B |
+
+
+
+### Effect of Different Data Sizes for Training BK-SDM-Small
+Increasing the number of training pairs improves the IS and CLIP scores over training progress. The MS-COCO 256×256 30K benchmark was used for evaluation.
+
+<center>
+    <img alt="Training progress with different data sizes" img src="https://netspresso-research-code-release.s3.us-east-2.amazonaws.com/assets-bk-sdm/fig_iter_data_size.png" width="100%">
+</center>
+
+Furthermore, with the growth in data volume, visual results become more favorable (e.g., better image-text alignment and clear distinction among objects). 
+
+<center>
+    <img alt="Visual results with different data sizes" img src="https://netspresso-research-code-release.s3.us-east-2.amazonaws.com/assets-bk-sdm/fig_results_data_size.png" width="100%">
+</center>
+
+### Additional Visual Examples
+
+<center>
+    <img alt="additional visual examples" img src="https://netspresso-research-code-release.s3.us-east-2.amazonaws.com/assets-bk-sdm/fig_results_models_2m.png" width="100%">
+</center>
+
+
+# Uses 
+Follow [the usage guidelines of Stable Diffusion v1](https://huggingface.co/CompVis/stable-diffusion-v1-4#uses). 
+
+
+# Acknowledgments
+- We express our gratitude to [Microsoft for Startups Founders Hub](https://www.microsoft.com/en-us/startups) for generously providing the Azure credits used during pretraining.
+- We deeply appreciate the pioneering research on Latent/Stable Diffusion conducted by [CompVis](https://github.com/CompVis/latent-diffusion), [Runway](https://runwayml.com/), and [Stability AI](https://stability.ai/).
+- Special thanks to the contributors to [LAION](https://laion.ai/), [Diffusers](https://github.com/huggingface/diffusers), and [Gradio](https://www.gradio.app/) for their valuable support.
+
+
+# Citation
+```bibtex
+@article{kim2023architectural,
+  title={On Architectural Compression of Text-to-Image Diffusion Models},
+  author={Kim, Bo-Kyeong and Song, Hyoung-Kyu and Castells, Thibault and Choi, Shinkook},
+  journal={arXiv preprint arXiv:2305.15798},
+  year={2023},
+  url={https://arxiv.org/abs/2305.15798}
+}
+```
+```bibtex
+@article{Kim_2023_ICMLW,
+  title={BK-SDM: Architecturally Compressed Stable Diffusion for Efficient Text-to-Image Generation},
+  author={Kim, Bo-Kyeong and Song, Hyoung-Kyu and Castells, Thibault and Choi, Shinkook},
+  journal={ICML Workshop on Efficient Systems for Foundation Models (ES-FoMo)},
+  year={2023},
+  url={https://openreview.net/forum?id=bOVydU0XKC}
+}
+```
+
+*This model card was written by Bo-Kyeong Kim and is based on the [Stable Diffusion v1 model card]( https://huggingface.co/CompVis/stable-diffusion-v1-4).*