TheBloke commited on
Commit
680d13d
1 Parent(s): 73fe156

Upload README.md

Browse files
Files changed (1) hide show
  1. README.md +483 -0
README.md ADDED
@@ -0,0 +1,483 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: LeoLM/leo-hessianai-13b-chat
3
+ datasets:
4
+ - LeoLM/OpenSchnabeltier
5
+ - OpenAssistant/OASST-DE
6
+ - FreedomIntelligence/alpaca-gpt4-deutsch
7
+ - FreedomIntelligence/evol-instruct-deutsch
8
+ - LeoLM/German_Poems
9
+ - LeoLM/German_Songs
10
+ inference: false
11
+ language:
12
+ - en
13
+ - de
14
+ library_name: transformers
15
+ license: llama2
16
+ model_creator: LAION LeoLM
17
+ model_name: Leo Hessianai 13B Chat
18
+ model_type: llama
19
+ pipeline_tag: text-generation
20
+ prompt_template: '<|im_start|>system
21
+
22
+ {system_message}<|im_end|>
23
+
24
+ <|im_start|>user
25
+
26
+ {prompt}<|im_end|>
27
+
28
+ <|im_start|>assistant
29
+
30
+ '
31
+ quantized_by: TheBloke
32
+ ---
33
+
34
+ <!-- header start -->
35
+ <!-- 200823 -->
36
+ <div style="width: auto; margin-left: auto; margin-right: auto">
37
+ <img src="https://i.imgur.com/EBdldam.jpg" alt="TheBlokeAI" style="width: 100%; min-width: 400px; display: block; margin: auto;">
38
+ </div>
39
+ <div style="display: flex; justify-content: space-between; width: 100%;">
40
+ <div style="display: flex; flex-direction: column; align-items: flex-start;">
41
+ <p style="margin-top: 0.5em; margin-bottom: 0em;"><a href="https://discord.gg/theblokeai">Chat & support: TheBloke's Discord server</a></p>
42
+ </div>
43
+ <div style="display: flex; flex-direction: column; align-items: flex-end;">
44
+ <p style="margin-top: 0.5em; margin-bottom: 0em;"><a href="https://www.patreon.com/TheBlokeAI">Want to contribute? TheBloke's Patreon page</a></p>
45
+ </div>
46
+ </div>
47
+ <div style="text-align:center; margin-top: 0em; margin-bottom: 0em"><p style="margin-top: 0.25em; margin-bottom: 0em;">TheBloke's LLM work is generously supported by a grant from <a href="https://a16z.com">andreessen horowitz (a16z)</a></p></div>
48
+ <hr style="margin-top: 1.0em; margin-bottom: 1.0em;">
49
+ <!-- header end -->
50
+
51
+ # Leo Hessianai 13B Chat - AWQ
52
+ - Model creator: [LAION LeoLM](https://huggingface.co/LeoLM)
53
+ - Original model: [Leo Hessianai 13B Chat](https://huggingface.co/LeoLM/leo-hessianai-13b-chat)
54
+
55
+ <!-- description start -->
56
+ ## Description
57
+
58
+ This repo contains AWQ model files for [LAION LeoLM's Leo Hessianai 13B Chat](https://huggingface.co/LeoLM/leo-hessianai-13b-chat).
59
+
60
+
61
+ ### About AWQ
62
+
63
+ AWQ is an efficient, accurate and blazing-fast low-bit weight quantization method, currently supporting 4-bit quantization. Compared to GPTQ, it offers faster Transformers-based inference.
64
+
65
+ It is also now supported by continuous batching server [vLLM](https://github.com/vllm-project/vllm), allowing use of Llama AWQ models for high-throughput concurrent inference in multi-user server scenarios.
66
+
67
+ As of September 25th 2023, preliminary Llama-only AWQ support has also been added to [Huggingface Text Generation Inference (TGI)](https://github.com/huggingface/text-generation-inference).
68
+
69
+ Note that, at the time of writing, overall throughput is still lower than running vLLM or TGI with unquantised models, however using AWQ enables using much smaller GPUs which can lead to easier deployment and overall cost savings. For example, a 70B model can be run on 1 x 48GB GPU instead of 2 x 80GB.
70
+ <!-- description end -->
71
+ <!-- repositories-available start -->
72
+ ## Repositories available
73
+
74
+ * [AWQ model(s) for GPU inference.](https://huggingface.co/TheBloke/leo-hessianai-13B-chat-AWQ)
75
+ * [GPTQ models for GPU inference, with multiple quantisation parameter options.](https://huggingface.co/TheBloke/leo-hessianai-13B-chat-GPTQ)
76
+ * [2, 3, 4, 5, 6 and 8-bit GGUF models for CPU+GPU inference](https://huggingface.co/TheBloke/leo-hessianai-13B-chat-GGUF)
77
+ * [LAION LeoLM's original unquantised fp16 model in pytorch format, for GPU inference and for further conversions](https://huggingface.co/LeoLM/leo-hessianai-13b-chat)
78
+ <!-- repositories-available end -->
79
+
80
+ <!-- prompt-template start -->
81
+ ## Prompt template: ChatML
82
+
83
+ ```
84
+ <|im_start|>system
85
+ {system_message}<|im_end|>
86
+ <|im_start|>user
87
+ {prompt}<|im_end|>
88
+ <|im_start|>assistant
89
+
90
+ ```
91
+
92
+ <!-- prompt-template end -->
93
+
94
+
95
+ <!-- README_AWQ.md-provided-files start -->
96
+ ## Provided files, and AWQ parameters
97
+
98
+ For my first release of AWQ models, I am releasing 128g models only. I will consider adding 32g as well if there is interest, and once I have done perplexity and evaluation comparisons, but at this time 32g models are still not fully tested with AutoAWQ and vLLM.
99
+
100
+ Models are released as sharded safetensors files.
101
+
102
+ | Branch | Bits | GS | AWQ Dataset | Seq Len | Size |
103
+ | ------ | ---- | -- | ----------- | ------- | ---- |
104
+ | [main](https://huggingface.co/TheBloke/leo-hessianai-13B-chat-AWQ/tree/main) | 4 | 128 | [German Quad](https://huggingface.co/datasets/deepset/germanquad) | 8192 | 7.25 GB
105
+
106
+ <!-- README_AWQ.md-provided-files end -->
107
+
108
+ <!-- README_AWQ.md-use-from-vllm start -->
109
+ ## Serving this model from vLLM
110
+
111
+ Documentation on installing and using vLLM [can be found here](https://vllm.readthedocs.io/en/latest/).
112
+
113
+ - When using vLLM as a server, pass the `--quantization awq` parameter, for example:
114
+
115
+ ```shell
116
+ python3 python -m vllm.entrypoints.api_server --model TheBloke/leo-hessianai-13B-chat-AWQ --quantization awq --dtype half
117
+ ```
118
+
119
+ Note: at the time of writing, vLLM has not yet done a new release with support for the `quantization` parameter.
120
+
121
+ If you try the code below and get an error about `quantization` being unrecognised, please install vLLM from Github source.
122
+
123
+ When using vLLM from Python code, pass the `quantization=awq` parameter, for example:
124
+
125
+ ```python
126
+ from vllm import LLM, SamplingParams
127
+
128
+ prompts = [
129
+ "Hello, my name is",
130
+ "The president of the United States is",
131
+ "The capital of France is",
132
+ "The future of AI is",
133
+ ]
134
+ sampling_params = SamplingParams(temperature=0.8, top_p=0.95)
135
+
136
+ llm = LLM(model="TheBloke/leo-hessianai-13B-chat-AWQ", quantization="awq", dtype="half")
137
+
138
+ outputs = llm.generate(prompts, sampling_params)
139
+
140
+ # Print the outputs.
141
+ for output in outputs:
142
+ prompt = output.prompt
143
+ generated_text = output.outputs[0].text
144
+ print(f"Prompt: {prompt!r}, Generated text: {generated_text!r}")
145
+ ```
146
+ <!-- README_AWQ.md-use-from-vllm start -->
147
+
148
+ <!-- README_AWQ.md-use-from-python start -->
149
+ ## Serving this model from TGI
150
+
151
+ TGI merged support for AWQ on September 25th, 2023. At the time of writing you need to use the `:latest` Docker container: `ghcr.io/huggingface/text-generation-inference:latest`
152
+
153
+ Add the parameter `--quantize awq` for AWQ support.
154
+
155
+ Example parameters:
156
+ ```shell
157
+ --model-id TheBloke/leo-hessianai-13B-chat-AWQ --port 3000 --quantize awq --max-input-length 3696 --max-total-tokens 4096 --max-batch-prefill-tokens 4096
158
+ ```
159
+
160
+ ## How to use this AWQ model from Python code
161
+
162
+ ### Install the necessary packages
163
+
164
+ Requires: [AutoAWQ](https://github.com/casper-hansen/AutoAWQ) 0.0.2 or later
165
+
166
+ ```shell
167
+ pip3 install autoawq
168
+ ```
169
+
170
+ If you have problems installing [AutoAWQ](https://github.com/casper-hansen/AutoAWQ) using the pre-built wheels, install it from source instead:
171
+
172
+ ```shell
173
+ pip3 uninstall -y autoawq
174
+ git clone https://github.com/casper-hansen/AutoAWQ
175
+ cd AutoAWQ
176
+ pip3 install .
177
+ ```
178
+
179
+ ### You can then try the following example code
180
+
181
+ ```python
182
+ from awq import AutoAWQForCausalLM
183
+ from transformers import AutoTokenizer
184
+
185
+ model_name_or_path = "TheBloke/leo-hessianai-13B-chat-AWQ"
186
+
187
+ # Load model
188
+ model = AutoAWQForCausalLM.from_quantized(model_name_or_path, fuse_layers=True,
189
+ trust_remote_code=False, safetensors=True)
190
+ tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, trust_remote_code=False)
191
+
192
+ prompt = "Tell me about AI"
193
+ prompt_template=f'''<|im_start|>system
194
+ {system_message}<|im_end|>
195
+ <|im_start|>user
196
+ {prompt}<|im_end|>
197
+ <|im_start|>assistant
198
+
199
+ '''
200
+
201
+ print("\n\n*** Generate:")
202
+
203
+ tokens = tokenizer(
204
+ prompt_template,
205
+ return_tensors='pt'
206
+ ).input_ids.cuda()
207
+
208
+ # Generate output
209
+ generation_output = model.generate(
210
+ tokens,
211
+ do_sample=True,
212
+ temperature=0.7,
213
+ top_p=0.95,
214
+ top_k=40,
215
+ max_new_tokens=512
216
+ )
217
+
218
+ print("Output: ", tokenizer.decode(generation_output[0]))
219
+
220
+ """
221
+ # Inference should be possible with transformers pipeline as well in future
222
+ # But currently this is not yet supported by AutoAWQ (correct as of September 25th 2023)
223
+ from transformers import pipeline
224
+
225
+ print("*** Pipeline:")
226
+ pipe = pipeline(
227
+ "text-generation",
228
+ model=model,
229
+ tokenizer=tokenizer,
230
+ max_new_tokens=512,
231
+ do_sample=True,
232
+ temperature=0.7,
233
+ top_p=0.95,
234
+ top_k=40,
235
+ repetition_penalty=1.1
236
+ )
237
+
238
+ print(pipe(prompt_template)[0]['generated_text'])
239
+ """
240
+ ```
241
+ <!-- README_AWQ.md-use-from-python end -->
242
+
243
+ <!-- README_AWQ.md-compatibility start -->
244
+ ## Compatibility
245
+
246
+ The files provided are tested to work with:
247
+
248
+ - [AutoAWQ](https://github.com/casper-hansen/AutoAWQ)
249
+ - [vLLM](https://github.com/vllm-project/vllm)
250
+ - [Huggingface Text Generation Inference (TGI)](https://github.com/huggingface/text-generation-inference)
251
+
252
+ TGI merged AWQ support on September 25th, 2023: [TGI PR #1054](https://github.com/huggingface/text-generation-inference/pull/1054). Use the `:latest` Docker container until the next TGI release is made.
253
+
254
+ <!-- README_AWQ.md-compatibility end -->
255
+
256
+ <!-- footer start -->
257
+ <!-- 200823 -->
258
+ ## Discord
259
+
260
+ For further support, and discussions on these models and AI in general, join us at:
261
+
262
+ [TheBloke AI's Discord server](https://discord.gg/theblokeai)
263
+
264
+ ## Thanks, and how to contribute
265
+
266
+ Thanks to the [chirper.ai](https://chirper.ai) team!
267
+
268
+ Thanks to Clay from [gpus.llm-utils.org](llm-utils)!
269
+
270
+ I've had a lot of people ask if they can contribute. I enjoy providing models and helping people, and would love to be able to spend even more time doing it, as well as expanding into new projects like fine tuning/training.
271
+
272
+ If you're able and willing to contribute it will be most gratefully received and will help me to keep providing more models, and to start work on new AI projects.
273
+
274
+ Donaters will get priority support on any and all AI/LLM/model questions and requests, access to a private Discord room, plus other benefits.
275
+
276
+ * Patreon: https://patreon.com/TheBlokeAI
277
+ * Ko-Fi: https://ko-fi.com/TheBlokeAI
278
+
279
+ **Special thanks to**: Aemon Algiz.
280
+
281
+ **Patreon special mentions**: Alicia Loh, Stephen Murray, K, Ajan Kanaga, RoA, Magnesian, Deo Leter, Olakabola, Eugene Pentland, zynix, Deep Realms, Raymond Fosdick, Elijah Stavena, Iucharbius, Erik Bjäreholt, Luis Javier Navarrete Lozano, Nicholas, theTransient, John Detwiler, alfie_i, knownsqashed, Mano Prime, Willem Michiel, Enrico Ros, LangChain4j, OG, Michael Dempsey, Pierre Kircher, Pedro Madruga, James Bentley, Thomas Belote, Luke @flexchar, Leonard Tan, Johann-Peter Hartmann, Illia Dulskyi, Fen Risland, Chadd, S_X, Jeff Scroggin, Ken Nordquist, Sean Connelly, Artur Olbinski, Swaroop Kallakuri, Jack West, Ai Maven, David Ziegler, Russ Johnson, transmissions 11, John Villwock, Alps Aficionado, Clay Pascal, Viktor Bowallius, Subspace Studios, Rainer Wilmers, Trenton Dambrowitz, vamX, Michael Levine, 준교 김, Brandon Frisco, Kalila, Trailburnt, Randy H, Talal Aujan, Nathan Dryer, Vadim, 阿明, ReadyPlayerEmma, Tiffany J. Kim, George Stoitzev, Spencer Kim, Jerry Meng, Gabriel Tamborski, Cory Kujawski, Jeffrey Morgan, Spiking Neurons AB, Edmond Seymore, Alexandros Triantafyllidis, Lone Striker, Cap'n Zoog, Nikolai Manek, danny, ya boyyy, Derek Yates, usrbinkat, Mandus, TL, Nathan LeClaire, subjectnull, Imad Khwaja, webtim, Raven Klaugh, Asp the Wyvern, Gabriel Puliatti, Caitlyn Gatomon, Joseph William Delisle, Jonathan Leane, Luke Pendergrass, SuperWojo, Sebastain Graf, Will Dee, Fred von Graf, Andrey, Dan Guido, Daniel P. Andersen, Nitin Borwankar, Elle, Vitor Caleffi, biorpg, jjj, NimbleBox.ai, Pieter, Matthew Berman, terasurfer, Michael Davis, Alex, Stanislav Ovsiannikov
282
+
283
+
284
+ Thank you to all my generous patrons and donaters!
285
+
286
+ And thank you again to a16z for their generous grant.
287
+
288
+ <!-- footer end -->
289
+
290
+ # Original model card: LAION LeoLM's Leo Hessianai 13B Chat
291
+
292
+ # LAION LeoLM: **L**inguistically **E**nhanced **O**pen **L**anguage **M**odel
293
+ Meet LeoLM, the first open and commercially available German Foundation Language Model built on Llama-2.
294
+ Our models extend Llama-2's capabilities into German through continued pretraining on a large corpus of German-language and mostly locality specific text.
295
+ Thanks to a compute grant at HessianAI's new supercomputer **42**, we release two foundation models trained with 8k context length,
296
+ [`LeoLM/leo-hessianai-7b`](https://huggingface.co/LeoLM/leo-hessianai-7b) and [`LeoLM/leo-hessianai-13b`](https://huggingface.co/LeoLM/leo-hessianai-13b) under the [Llama-2 community license](https://huggingface.co/meta-llama/Llama-2-70b/raw/main/LICENSE.txt) (70b also coming soon! 👀).
297
+ With this release, we hope to bring a new wave of opportunities to German open-source and commercial LLM research and accelerate adoption.
298
+ Read our [blog post]() or our paper (preprint coming soon) for more details!
299
+
300
+ *A project by Björn Plüster and Christoph Schuhmann in collaboration with LAION and HessianAI.*
301
+
302
+ ## LeoLM Chat
303
+ `LeoLM/leo-hessianai-13b-chat` is a German chat model built on our foundation model `LeoLM/leo-hessianai-13b` and finetuned on a selection of German instruction datasets.
304
+ The model performs exceptionally well on writing, explanation and discussion tasks but struggles somewhat with math and advanced reasoning. See our MT-Bench-DE scores:
305
+ ```
306
+ {
307
+ "first_turn": 6.525,
308
+ "second_turn": 5.15,
309
+ "categories": {
310
+ "writing": 6.925,
311
+ "roleplay": 6.7,
312
+ "reasoning": 4.55,
313
+ "math": 3.25,
314
+ "coding": 3.45,
315
+ "extraction": 5.4,
316
+ "stem": 7.55,
317
+ "humanities": 8.875
318
+ },
319
+ "average": 5.8375
320
+ }
321
+ ```
322
+
323
+ ## Model Details
324
+
325
+ - **Finetuned from:** [LeoLM/leo-hessianai-13b](https://huggingface.co/LeoLM/leo-hessianai-7b)
326
+ - **Model type:** Causal decoder-only transformer language model
327
+ - **Language:** English and German
328
+ - **Demo:** [Web Demo]()
329
+ - **License:** [LLAMA 2 COMMUNITY LICENSE AGREEMENT](https://huggingface.co/meta-llama/Llama-2-70b/raw/main/LICENSE.txt)
330
+ - **Contact:** [LAION Discord](https://discord.com/invite/eq3cAMZtCC) or [Björn Plüster](mailto:bjoern.pl@outlook.de)
331
+
332
+
333
+ ## Use in 🤗Transformers
334
+ First install direct dependencies:
335
+ ```
336
+ pip install transformers torch sentencepiece
337
+ ```
338
+ If you want faster inference using flash-attention2, you need to install these dependencies:
339
+ ```bash
340
+ pip install packaging ninja
341
+ pip install flash-attn==v2.1.1 --no-build-isolation
342
+ pip install git+https://github.com/HazyResearch/flash-attention.git@v2.1.1#subdirectory=csrc/rotary
343
+ ```
344
+ Then load the model in transformers:
345
+ ```python
346
+ from transformers import pipeline
347
+ import torch
348
+
349
+ system_prompt = """<|im_start|>system
350
+ Dies ist eine Unterhaltung zwischen einem intelligenten, hilfsbereitem KI-Assistenten und einem Nutzer.
351
+ Der Assistent gibt ausführliche, hilfreiche und ehrliche Antworten.<|im_end|>
352
+
353
+ """
354
+ prompt_format = "<|im_start|>user\n{prompt}<|im_end|>\n<|im_start|>assistant\n"
355
+ prompt = "Erkläre mir wie die Fahrradwegesituation in Hamburg ist."
356
+
357
+ generator = pipeline(model="LeoLM/leo-hessianai-13b-chat", device="cuda", torch_dtype=torch.float16, trust_remote_code=True) # True for flash-attn2 else False
358
+ print(generator(prompt_format.format(prompt=prompt), do_sample=True, top_p=0.95, max_length=8192))
359
+ ```
360
+
361
+ "*Hamburg verfügt über ein umfangreiches Netzwerk an Radwegen, das ein effizientes und angenehmes Radfahren in der ganzen Stadt ermöglicht. Die Radwege variieren in Bezug auf ihre Qualität, wobei einige erstklassig mit eigens für Radfahrer reservierten Spuren sind, während andere einfache Fahrradsymbole auf dem Boden haben, die anzeigen, dass Radfahrer abwechselnd mit dem Autoverkehr auf der Straße fahren sollten. Einige Nebenstraßen haben auch spezielle Fahrradspuren, wobei einige mit Bordsteinabsenkungen zur Seite der Autospuren markiert sind. Zusätzlich haben viele Hauptstraßen, insbesondere in Nebenstraßen, fahrradfreundliche Abstände zwischen den geparkten Autos und dem Gehweg, was ein bequemes Fahren auf der Straße ermöglicht. Der Bau von Radschnellwegen, die schnelles und effizientes Radfahren in und aus der Stadt ermöglichen, ist im Gange und wird in den kommenden Jahren fortgesetzt. Insgesamt sind die Radwege in Hamburg weitläufig und gut ausgeschildert, was es zu einem angenehmen Ort macht, um mit dem Fahrrad zu fahren.*"
362
+
363
+ ## Prompting / Prompt Template
364
+
365
+ Prompt dialogue template (ChatML format):
366
+
367
+ ```
368
+ """
369
+ <|im_start|>system
370
+ {system_message}<|im_end|>
371
+ <|im_start|>user
372
+ {prompt}<|im_end|>
373
+ <|im_start|>assistant
374
+ """
375
+ ```
376
+
377
+ The model input can contain multiple conversation turns between user and assistant, e.g.
378
+ ```
379
+ <|im_start|>user
380
+ {prompt 1}<|im_end|>
381
+ <|im_start|>assistant
382
+ {reply 1}<|im_end|>
383
+ <|im_start|>user
384
+ {prompt 2}<|im_end|>
385
+ <|im_start|>assistant
386
+ (...)
387
+ ```
388
+
389
+ ## Ethical Considerations and Limitations
390
+
391
+ LeoLM has been tested in English and German, and has not covered, nor could it cover all scenarios.
392
+ For these reasons, as with all LLMs, the potential outputs of `LeoLM/leo-hessianai-13b-chat` cannot be predicted
393
+ in advance, and the model may in some instances produce inaccurate, biased or other objectionable responses
394
+ to user prompts. Therefore, before deploying any applications of `LeoLM/leo-hessianai-13b-chat`, developers should
395
+ perform safety testing and tuning tailored to their specific applications of the model.
396
+
397
+ Please see Meta's [Responsible Use Guide](https://ai.meta.com/llama/responsible-use-guide/).
398
+
399
+ ## Finetuning Details
400
+
401
+ | Hyperparameter | Value |
402
+ |---|---|
403
+ | Num epochs | 3 |
404
+ | Examples per epoch | 131214 |
405
+ | Global batch size | 256 |
406
+ | Learning rate | 3e-5 |
407
+ | Warmup steps | 100 |
408
+ | LR scheduler | Cosine |
409
+ | Adam betas | (0.9, 0.95) |
410
+
411
+
412
+ ## Dataset Details
413
+ ```
414
+ ## Stats for 'Subset of OpenAssistant/OASST-DE' (3534 samples (100.0%))
415
+ -----------------
416
+ Accepted: 3534/3534 (100.0%)
417
+ Accepted tokens: 2259302
418
+ Skipped: 0 (0.0%)
419
+ Min tokens per sample: 29
420
+ Max tokens per sample: 2484
421
+ Avg tokens per sample: 639.3044708545557
422
+ -----------------
423
+
424
+ ## Stats for 'Subset of FreedomIntelligence/evol-instruct-deutsch' (57841 samples (100.0%))
425
+ -----------------
426
+ Accepted: 57841/57841 (100.0%)
427
+ Accepted tokens: 42958192
428
+ Skipped: 0 (0.0%)
429
+ Min tokens per sample: 33
430
+ Max tokens per sample: 5507
431
+ Avg tokens per sample: 742.6944900675991
432
+ -----------------
433
+
434
+ ## Stats for 'Subset of FreedomIntelligence/alpaca-gpt4-deutsch' (48969 samples (100.0%))
435
+ -----------------
436
+ Accepted: 48969/48969 (100.0%)
437
+ Accepted tokens: 13372005
438
+ Skipped: 0 (0.0%)
439
+ Min tokens per sample: 19
440
+ Max tokens per sample: 1359
441
+ Avg tokens per sample: 273.07082031489307
442
+ -----------------
443
+
444
+ ## Stats for 'Subset of LeoLM/OpenSchnabeltier' (21314 samples (100.0%))
445
+ -----------------
446
+ Accepted: 21314/21314 (100.0%)
447
+ Accepted tokens: 8134690
448
+ Skipped: 0 (0.0%)
449
+ Min tokens per sample: 25
450
+ Max tokens per sample: 1202
451
+ Avg tokens per sample: 381.65947264708643
452
+ -----------------
453
+
454
+ ## Stats for 'Subset of LeoLM/German_Poems' (490 samples (100.0%))
455
+ -----------------
456
+ Accepted: 490/490 (100.0%)
457
+ Accepted tokens: 618642
458
+ Skipped: 0 (0.0%)
459
+ Min tokens per sample: 747
460
+ Max tokens per sample: 1678
461
+ Avg tokens per sample: 1262.534693877551
462
+ -----------------
463
+
464
+ ## Stats for 'Subset of LeoLM/German_Songs' (392 samples (100.0%))
465
+ -----------------
466
+ Accepted: 392/392 (100.0%)
467
+ Accepted tokens: 187897
468
+ Skipped: 0 (0.0%)
469
+ Min tokens per sample: 231
470
+ Max tokens per sample: 826
471
+ Avg tokens per sample: 479.3290816326531
472
+ -----------------
473
+
474
+ ## Stats for 'total' (132540 samples (100.0%))
475
+ -----------------
476
+ Accepted: 132540/132540 (100.0%)
477
+ Accepted tokens: 67530728
478
+ Skipped: 0 (0.0%)
479
+ Min tokens per sample: 19
480
+ Max tokens per sample: 5507
481
+ Avg tokens per sample: 509.51205673758864
482
+ -----------------
483
+ ```