SinclairWang commited on
Commit
2936274
1 Parent(s): 326aabb

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +7 -6
README.md CHANGED
@@ -18,16 +18,17 @@ base_model:
18
 
19
  [ArXiv](http://arxiv.org/abs/xxxx) | [Data: OpenWebMath-Pro](https://huggingface.co/datasets/gair-prox/open-web-math-pro) | [Code](https://github.com/GAIR-NLP/program-every-example)
20
 
21
- **CodeLlama-7B-ProXMath** is a language model that is continually pre-trained on [OpenWebMath-Pro](https://huggingface.co/datasets/gair-prox/open-web-math-pro) (a refined version by ProX) for 10B tokens.
22
 
23
  ## Evaluations
24
 
25
- ProX models are evaluated over 10 language model benchmarks in zero-shot setting.
 
 
 
 
 
26
 
27
- | | ArC-c | ARC-e | CSQA | HellaS | MMLU | OBQA | PiQA | SIQA | WinoG | SciQ | AVG |
28
- |-----------------------|-------|-------|-------|-----------|-------|-------|-------|-------|-------|-------|------|
29
- | raw | 26.1 | 44.3 | 29.7 | 39.1 | 27.3 | 29.2 | 66.9 | 39.0 | 52.0 | 67.4 | 42.1 |
30
- | ours | 26.4 | 51.9 | 30.9 | 42.4 | 29.4 | 31.6 | 67.9 | 40.0 | 52.2 | 73.5 | 44.6 |
31
 
32
  ### Citation
33
  ```
 
18
 
19
  [ArXiv](http://arxiv.org/abs/xxxx) | [Data: OpenWebMath-Pro](https://huggingface.co/datasets/gair-prox/open-web-math-pro) | [Code](https://github.com/GAIR-NLP/program-every-example)
20
 
21
+ **CodeLlama-7B-ProXMath** is a language model that is continually pre-trained on [OpenWebMath-Pro](https://huggingface.co/datasets/gair-prox/open-web-math-pro) (a refined version by ProX) for **10**B tokens.
22
 
23
  ## Evaluations
24
 
25
+ ProX models are evaluated on 9 common math reasoning benchmarks.
26
+
27
+ | Model | asdiv | gsm8k | mathqa | mawps | minerva_math | mmlu_stem | sat_math | svamp | tabmwp | average |
28
+ |----------------------- |---------- |---------- |---------- |---------- |-------------- |----------- |---------- |---------- |---------- |---------- |
29
+ | CodeLlama-7B | 50.7 | 11.8 | 14.3 | 62.6 | 5.0 | 20.4 | 21.9 | 44.2 | 30.6 | 29.1 |
30
+ | CodeLlama-7B-ProXMath | **67.9** | **35.6** | **38.9** | **82.7** | **17.6** | **42.6** | **62.5** | **55.8** | **41.3** | **49.4** |
31
 
 
 
 
 
32
 
33
  ### Citation
34
  ```