---
datasets:
- SirNeural/flan_v2
metrics:
- perplexity
tags:
- flan
- opt
- peft
---

## ptune-FLAN-OPT-2.7b

facebook/opt-2.7b finetuned with prefix tuning (https://arxiv.org/abs/2101.00190) with the FLAN datasets (https://arxiv.org/pdf/2210.11416.pdf).

24 token prefix finetuned over 3.7m new tokens of a FLAN task mixture.

It reaches a train ppl of 5.95 and an eval ppl of 4.50

### Example COT (Chain-of-thought) Prompt:

```
Q: Answer the following yes/no question by reasoning step-by-step. Could a dandelion suffer from hepatitis?
A: Hepatitis only affects organisms with livers. Dandelions don’t have a liver. The answer is no.

Q: Answer the following yes/no question by reasoning step-by-step. Can you write a whole Haiku in a single tweet?
A: A haiku is a japanese three-line poem. That is short enough to fit in 280 characters. The answer is yes.

Q: Answer the following yes/no question by reasoning step-by-step. Can you reach space with a Cessna?
A:
```
<p style='color:green'> > Cessna is a type of aircraft. It has a wingspan of 3.6 meters (3.6m). The answer is no.</p>