arnocandel commited on
Commit
19cfca2
1 Parent(s): d5d0b9d

commit files to HF hub

Browse files
Files changed (1) hide show
  1. README.md +17 -15
README.md CHANGED
@@ -13,6 +13,8 @@ tags:
13
  datasets:
14
  - h2oai/h2ogpt-oig-oasst1-instruct-cleaned-v1
15
  - h2oai/openassistant_oasst1_h2ogpt
 
 
16
  ---
17
  # h2oGPT Model Card
18
  ## Summary
@@ -20,9 +22,9 @@ datasets:
20
  H2O.ai's `h2ogpt-oig-oasst1-512-6.9b` is a 6.9 billion parameter instruction-following large language model licensed for commercial use.
21
 
22
  - Base model: [EleutherAI/pythia-6.9b](https://huggingface.co/EleutherAI/pythia-6.9b)
23
- - Fine-tuning dataset: [h2oai/h2ogpt-oig-oasst1-instruct-cleaned-v1](https://huggingface.co/datasets/h2oai/h2ogpt-oig-oasst1-instruct-cleaned-v1) and [h2oai/openassistant_oasst1_h2ogpt](https://huggingface.co/datasets/h2oai/openassistant_oasst1_h2ogpt)
24
  - Data-prep and fine-tuning code: [H2O.ai GitHub](https://github.com/h2oai/h2ogpt)
25
- - Training logs: [zip](https://huggingface.co/h2oai/h2ogpt-oig-oasst1-512-6.9b/blob/main/pythia-6.9b.h2ogpt-oig-oasst1-instruct-cleaned-v1.json.1_epochs.5fc91911bc2bfaaf3b6c2de577c4b0ae45a07a4a.7.zip) and [zip](https://huggingface.co/h2oai/h2ogpt-oig-oasst1-512-6.9b/blob/main/h2ogpt-oig-oasst1-512-6.9b.h2oaiopenassistant_oasst1_h2ogpt.2_epochs.e35e2e06e0af2f7dceac2e16e3646c90ccce4ec0.1.zip)
26
 
27
  ## Chatbot
28
 
@@ -134,22 +136,22 @@ GPTNeoXConfig {
134
  Model validation results using [EleutherAI lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness).
135
 
136
 
137
- [eval source code](https://github.com/h2oai/h2ogpt/issues/35#issuecomment-1520876120)
138
 
139
  | Task |Version| Metric |Value | |Stderr|
140
  |-------------|------:|--------|-----:|---|-----:|
141
- |boolq | 1|acc |0.6266|± |0.0085|
142
- |arc_challenge| 0|acc |0.3225|± |0.0137|
143
- | | |acc_norm|0.3396|± |0.0138|
144
- |openbookqa | 0|acc |0.2660|± |0.0198|
145
- | | |acc_norm|0.3660|± |0.0216|
146
- |arc_easy | 0|acc |0.6776|± |0.0096|
147
- | | |acc_norm|0.6195|± |0.0100|
148
- |hellaswag | 0|acc |0.4822|± |0.0050|
149
- | | |acc_norm|0.6465|± |0.0048|
150
- |winogrande | 0|acc |0.6219|± |0.0136|
151
- |piqa | 0|acc |0.7530|± |0.0101|
152
- | | |acc_norm|0.7606|± |0.0100|
153
 
154
 
155
  ## Disclaimer
 
13
  datasets:
14
  - h2oai/h2ogpt-oig-oasst1-instruct-cleaned-v1
15
  - h2oai/openassistant_oasst1_h2ogpt
16
+ - h2oai/h2ogpt-fortune2000-personalized
17
+ - h2oai/h2ogpt-oig-oasst1-instruct-cleaned-v3
18
  ---
19
  # h2oGPT Model Card
20
  ## Summary
 
22
  H2O.ai's `h2ogpt-oig-oasst1-512-6.9b` is a 6.9 billion parameter instruction-following large language model licensed for commercial use.
23
 
24
  - Base model: [EleutherAI/pythia-6.9b](https://huggingface.co/EleutherAI/pythia-6.9b)
25
+ - Fine-tuning dataset: [h2oai/h2ogpt-oig-oasst1-instruct-cleaned-v1](https://huggingface.co/datasets/h2oai/h2ogpt-oig-oasst1-instruct-cleaned-v1) and [h2oai/openassistant_oasst1_h2ogpt](https://huggingface.co/datasets/h2oai/openassistant_oasst1_h2ogpt) and [h2oai/h2ogpt-fortune2000-personalized](https://huggingface.co/datasets/h2oai/h2ogpt-fortune2000-personalized) and [h2oai/h2ogpt-oig-oasst1-instruct-cleaned-v3](https://huggingface.co/datasets/h2oai/h2ogpt-oig-oasst1-instruct-cleaned-v3)
26
  - Data-prep and fine-tuning code: [H2O.ai GitHub](https://github.com/h2oai/h2ogpt)
27
+ - Training logs: [zip](https://huggingface.co/h2oai/h2ogpt-oig-oasst1-512-6.9b/blob/main/pythia-6.9b.h2ogpt-oig-oasst1-instruct-cleaned-v1.json.1_epochs.5fc91911bc2bfaaf3b6c2de577c4b0ae45a07a4a.7.zip) and [zip](https://huggingface.co/h2oai/h2ogpt-oig-oasst1-512-6.9b/blob/main/h2ogpt-oig-oasst1-512-6.9b.h2oaiopenassistant_oasst1_h2ogpt.2_epochs.e35e2e06e0af2f7dceac2e16e3646c90ccce4ec0.1.zip) and [zip](https://huggingface.co/h2oai/h2ogpt-oig-oasst1-512-6.9b/blob/main/h2ogpt-oig-oasst1-512-6.9b.h2oaih2ogpt-oig-oasst1-instruct-cleaned-v3.1_epochs.e48f9debb0d2bd8d866fa5668bbbb51c317c553c.1.zip)
28
 
29
  ## Chatbot
30
 
 
136
  Model validation results using [EleutherAI lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness).
137
 
138
 
139
+ [eval source code](https://github.com/h2oai/h2ogpt/issues/125#issue-1702311702)
140
 
141
  | Task |Version| Metric |Value | |Stderr|
142
  |-------------|------:|--------|-----:|---|-----:|
143
+ |arc_easy | 0|acc |0.6591|± |0.0097|
144
+ | | |acc_norm|0.6178|± |0.0100|
145
+ |arc_challenge| 0|acc |0.3174|± |0.0136|
146
+ | | |acc_norm|0.3558|± |0.0140|
147
+ |openbookqa | 0|acc |0.2540|± |0.0195|
148
+ | | |acc_norm|0.3580|± |0.0215|
149
+ |winogrande | 0|acc |0.6069|± |0.0137|
150
+ |piqa | 0|acc |0.7486|± |0.0101|
151
+ | | |acc_norm|0.7546|± |0.0100|
152
+ |hellaswag | 0|acc |0.4843|± |0.0050|
153
+ | | |acc_norm|0.6388|± |0.0048|
154
+ |boolq | 1|acc |0.6193|± |0.0085|
155
 
156
 
157
  ## Disclaimer