README.md

---
language:
- ko
datasets:
- DopeorNope/DPO-Ko-Dataset
- DopeorNope/Orca_Near_Dedup-v2
library_name: transformers
pipeline_tag: text-generation
license: cc-by-nc-sa-4.0
---

**(주)미디어그룹사람과숲과 (주)마커의 LLM 연구 컨소시엄으로 개발된 모델입니다**


**The license is `cc-by-nc-sa-4.0`.**  
  
# **🐻‍❄️COKAL-DPO_13b-v2🐻‍❄️**  

![img](https://drive.google.com/uc?export=view&id=1YGBxz-UhQGHZ2K6cTXmTnB13fRgaQilX)  

## Model Details

**Model Developers** Seungyoo Lee (DopeorNope)




**Input** Models input text only.

**Output** Models generate text only.

**Model Architecture**  
COKAL-DPO_13b-v2 is an auto-regressive 13B language model based on the LLaMA2 transformer architecture.

**Base Model**  [DopeorNope/COKAL_pre_DPO_Test_v2-13b](https://huggingface.co/DopeorNope/COKAL_pre_DPO_Test_v2-13b)   

DopeorNope/COKAL_pre_DPO_Test_v2-13b is the SFT model to train with DPO methodology.

**Training Dataset**  
- DPO training dataset: [DopeorNope/DPO-Ko-Dataset](private) - private

This dataset was constructed by directly collecting and reorganizing data by DopeorNope, obtaining insights from ["lvwerra/stack-exchange-paired"](https://huggingface.co/datasets/lvwerra/stack-exchange-paired) to create a paired dataset. (It means I do not use stack-exchange-paired; I just got an insight from it.)

- SFT training dataset: [DopeorNope/Orca_Near_Dedup-v2](private) - private

This dataset is based on ["kyujinpy/OpenOrca-KO"](https://huggingface.co/datasets/kyujinpy/OpenOrca-KO) and has been processed using the Near Dedup algorithm to remove items with a Jaccard Similarity threshold of 0.8 or higher. In addition, inconsistent inputs have been cleaned and modified.
  
**Training**  
The difference between "DopeorNope/COKAL-DPO_test-v2" and this model is that this model has different hyper-parameters from the one in that setting regarding the final version.

I developed the model in an environment with four RTX 3090 GPUs running Ubuntu 18.04. 

It seems that when uploading the model directly to a repository from a Linux server, there may be an issue causing the model to appear to have more parameters. However, this model is based on a 13B architecture.


**Reference papers**

- Data Strategy:
  - [LIMA(Zhou et al., 2023)](https://arxiv.org/abs/2305.11206)
  - [Near Dedup algorithm(Lee et al., 2022)](https://arxiv.org/abs/2107.06499)

- Model Architecture:
  - [Llama2(Touvron et al., 2023)](https://arxiv.org/abs/2307.09288)


# Implementation Code
```python

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

repo = "HumanF-MarkrAI/COKAL-DPO-13b-v2"
model = AutoModelForCausalLM.from_pretrained(
        repo,
        return_dict=True,
        torch_dtype=torch.float16,
        device_map='auto'
)
model_tokenizer = AutoTokenizer.from_pretrained(repo)
```


# Acknowledgement

- 이 모델은 과학기술정보통신부·광주광역시가 공동 지원한 '인공지능 중심 산업융합 집적단지 조성사업'으로 지원을 받아 수행된 연구 결과입니다.

- This model was supported by Artificial intelligence industrial convergence cluster development project funded by the Ministry of Science and ICT(MSIT, Korea)&Gwangju Metropolitan City.



---