File size: 358 Bytes
3eab653 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
---
datasets:
- SirNeural/flan_v2
metrics:
- perplexity
tags:
- flan
- opt
- peft
---
facebook/opt-2.7b finetuned with prefix tuning (https://arxiv.org/abs/2101.00190) with the FLAN datasets (https://arxiv.org/pdf/2210.11416.pdf).
24 token prefix finetuned over 3.7m new tokens of a FLAN task mixture.
It reaches a train ppl of 5.95 and an eval ppl of 4.50 |