a collection of pre-training corpora refined by ProX
GAIR-ProX
community
AI & ML interests
NLP Research
Organization Card
GAIR-ProX, a subsidiary of GAIR, spearheads the 🫐 ProX Project. This initiative aims to enhance pre-training efficiency by refining corpus documents using language models at scale. Through meticulous operations (e.g., document-level filtering and chunk-level cleaning), implemented as scalable, executable programs, 🫐 ProX seeks to improve pre-training data quality at scale, ultimately developing more robust and efficient language models.
Read our technical report!
models
10
gair-prox/FW-ProX-1.7B
Text Generation
•
Updated
•
9
•
2
gair-prox/Mistral-7B-ProXMath
Text Generation
•
Updated
•
11
•
2
gair-prox/Llama-2-7B-ProXMath
Text Generation
•
Updated
•
4
gair-prox/TinyLlama-1.1B-ProXMath
Updated
•
2
gair-prox/CodeLlama-7B-ProXMath
Updated
•
7
gair-prox/RedPJ-ProX-0.3B
Updated
•
4
gair-prox/C4-ProX-1.7B
Updated
•
4
gair-prox/RedPJ-ProX-1.7B
Updated
•
4
gair-prox/RedPJ-ProX-0.7B
Updated
•
4
gair-prox/ProX-RedPJ-1.7B-25B
Updated
•
4