Blaze7451 commited on
Commit
6f8634a
1 Parent(s): 7d2ac7e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +15 -1
README.md CHANGED
@@ -1,5 +1,10 @@
1
  ---
2
  license: llama2
 
 
 
 
 
3
  ---
4
  # Bailong-orpo 7B
5
  <div align="center">
@@ -18,7 +23,16 @@ This model card contains the information and the results of our released Bailong
18
  - **Bailong-bench:** Most existing language models claiming to support Traditional Chinese are adapted from continuously pre-trained open-source models, primarily trained on English data. In certain cases, models fine-tuned with instructions using this approach may respond to Traditional Chinese instructions in English and vice versa. This could pose a significant problem when deploying the model for real-world applications. Consequently, it is essential to have a benchmark dataset specifically designed to assess a model's proficiency in following both English and Traditional Chinese instructions. To address this issue, we propose Bailong-bench, a benchmark dataset crafted not only to evaluate the model's performance in various real-world application scenarios but also to assess its ability to maintain language consistency.
19
  - **Technical report:** In our [technical report](https://arxiv.org/abs/2404.00862), we document the model training process and the details regarding the sources of training data.
20
  - **Bailong-orpo 7B:** Leveraging monolithic odds ratio preference optimization
21
- algorithm, [ORPO](https://arxiv.org/abs/2403.07691), we further fine-tune Bailong-instruct 7B with 180k preference training data to derive Bailong-orpo 7B.
 
 
 
 
 
 
 
 
 
22
 
23
  ## Applications (Bailong-orpo 7B)
24
  The following tables present, but are not limited to, several possible scenarios for the applications of Bailong-orpo 7B. All the following model outputs are generated under the same generation configuration (temperature=0.6, top-p=0.9, top-k=40, repetition_penalty=1.1)
 
1
  ---
2
  license: llama2
3
+ library_name: transformers
4
+ base_model: INX-TEXT/Bailong-instruct-7B
5
+ pipeline_tag: text-generation
6
+ tags:
7
+ - orpo
8
  ---
9
  # Bailong-orpo 7B
10
  <div align="center">
 
23
  - **Bailong-bench:** Most existing language models claiming to support Traditional Chinese are adapted from continuously pre-trained open-source models, primarily trained on English data. In certain cases, models fine-tuned with instructions using this approach may respond to Traditional Chinese instructions in English and vice versa. This could pose a significant problem when deploying the model for real-world applications. Consequently, it is essential to have a benchmark dataset specifically designed to assess a model's proficiency in following both English and Traditional Chinese instructions. To address this issue, we propose Bailong-bench, a benchmark dataset crafted not only to evaluate the model's performance in various real-world application scenarios but also to assess its ability to maintain language consistency.
24
  - **Technical report:** In our [technical report](https://arxiv.org/abs/2404.00862), we document the model training process and the details regarding the sources of training data.
25
  - **Bailong-orpo 7B:** Leveraging monolithic odds ratio preference optimization
26
+ algorithm, [ORPO](https://arxiv.org/abs/2403.07691), we further fine-tune Bailong-instruct 7B with 180k preference pair data to derive Bailong-orpo 7B. We also provide f16 GGUF version of Bailong-orpo 7B for efficient inference and storage purposes.
27
+
28
+ ## Model information
29
+ - License: [Llama-2 License](https://ai.meta.com/llama/license/)
30
+ - Base Model: [INX-TEXT/Bailong-instruct-7B](https://huggingface.co/INX-TEXT/Bailong-instruct-7B)
31
+ - Type: decoder-only transformer architecture
32
+ - Model Size: 6.96B
33
+ - Context length: 2048
34
+ - Vocabulary size: 59241
35
+ - Language: English and Traditional Chinese
36
 
37
  ## Applications (Bailong-orpo 7B)
38
  The following tables present, but are not limited to, several possible scenarios for the applications of Bailong-orpo 7B. All the following model outputs are generated under the same generation configuration (temperature=0.6, top-p=0.9, top-k=40, repetition_penalty=1.1)