Commit
•
24c6af7
1
Parent(s):
959320a
commit files to HF hub
Browse files
README.md
CHANGED
|
@@ -144,9 +144,23 @@ RWConfig {
|
|
| 144 |
Model validation results using [EleutherAI lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness).
|
| 145 |
|
| 146 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 147 |
|
| 148 |
-
TBD
|
| 149 |
-
|
| 150 |
|
| 151 |
## Disclaimer
|
| 152 |
|
|
|
|
| 144 |
Model validation results using [EleutherAI lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness).
|
| 145 |
|
| 146 |
|
| 147 |
+
[eval source code](https://github.com/h2oai/h2ogpt/issues/216#issuecomment-1579573101)
|
| 148 |
+
|
| 149 |
+
| Task |Version| Metric |Value | |Stderr|
|
| 150 |
+
|-------------|------:|--------|-----:|---|-----:|
|
| 151 |
+
|arc_challenge| 0|acc |0.4957|± |0.0146|
|
| 152 |
+
| | |acc_norm|0.5324|± |0.0146|
|
| 153 |
+
|arc_easy | 0|acc |0.8140|± |0.0080|
|
| 154 |
+
| | |acc_norm|0.7837|± |0.0084|
|
| 155 |
+
|boolq | 1|acc |0.8297|± |0.0066|
|
| 156 |
+
|hellaswag | 0|acc |0.6490|± |0.0048|
|
| 157 |
+
| | |acc_norm|0.8293|± |0.0038|
|
| 158 |
+
|openbookqa | 0|acc |0.3780|± |0.0217|
|
| 159 |
+
| | |acc_norm|0.4740|± |0.0224|
|
| 160 |
+
|piqa | 0|acc |0.8248|± |0.0089|
|
| 161 |
+
| | |acc_norm|0.8362|± |0.0086|
|
| 162 |
+
|winogrande | 0|acc |0.7837|± |0.0116|
|
| 163 |
|
|
|
|
|
|
|
| 164 |
|
| 165 |
## Disclaimer
|
| 166 |
|