---
license: cc-by-nc-4.0
datasets:
- vicgalle/alpaca-gpt4
language:
- en
---

## Model Overview
Model license: cc-by-nc-4.0<br>
This model is trained based on [EleutherAI/pythia-1.4b-deduped](https://huggingface.co/EleutherAI/pythia-1.4b-deduped) model that is LoRA finetuned on [vicgalle/alpaca-gpt4](https://huggingface.co/datasets/vicgalle/alpaca-gpt4) dataset.<br>

## Prompt Template: `Alpaca`
```
<system_prompt>

### Instruction:
<user_message>

### Response:
<assistant_response>
```

## Intended Use
THIS IS A TEST MODEL, IT IS NOT INTENDED FOR REAL APPLICATIONS BY ANY MEANS. HOWEVER, A NEW MODEL IS COMING IN THE SAME TOPIC.<br>
This model series will be used for small but intense applications.

## Training Details
This model took `2:31:23` to train in QLoRA on a single  `T4`  GPU.<br>
 - *epochs*:  `1`
 - *train batch size*:  `12`
 - *eval batch size*:  `12`
 - *gradient accumulation steps*:  `1`
 - *maximum gradient normal*:  `0.3`
 - *learning rate*:  `2e-4`
 - *weight decay*:  `0.001`
 - *optimizer*:  `paged_adamw_32bit`
 - *learning rate schedule*:  `cosine`
 - *warmup ratio (linear)*:  `0.03`