Create README.md

README.md

@@ -0,0 +1,15 @@
+---
+datasets:
+- SirNeural/flan_v2
+metrics:
+- perplexity
+tags:
+- flan
+- opt
+- peft
+---
+facebook/opt-2.7b finetuned with prefix tuning (https://arxiv.org/abs/2101.00190) with the FLAN datasets (https://arxiv.org/pdf/2210.11416.pdf).
+
+24 token prefix finetuned over 3.7m new tokens of a FLAN task mixture.
+
+It reaches a train ppl of 5.95 and an eval ppl of 4.50