metadata
license: cc-by-nc-4.0
datasets:
- vicgalle/alpaca-gpt4
language:
- en
Model Overview
Model license: cc-by-nc-4.0
This model is trained based on EleutherAI/pythia-1.4b-deduped model that is LoRA finetuned on vicgalle/alpaca-gpt4 dataset.
Prompt Template: Alpaca
<system_prompt>
### Instruction:
<user_message>
### Response:
<assistant_response>
Intended Use
THIS IS A TEST MODEL, IT IS NOT INTENDED FOR REAL APPLICATIONS BY ANY MEANS. HOWEVER, A NEW MODEL IS COMING IN THE SAME TOPIC.
This model series will be used for small but intense applications.
Training Details
This model took 2:31:23
to train in QLoRA on a single T4
GPU.
- epochs:
1
- train batch size:
12
- eval batch size:
12
- gradient accumulation steps:
1
- maximum gradient normal:
0.3
- learning rate:
2e-4
- weight decay:
0.001
- optimizer:
paged_adamw_32bit
- learning rate schedule:
cosine
- warmup ratio (linear):
0.03
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
Metric | Value |
---|---|
Avg. | 31.17 |
ARC (25-shot) | 34.22 |
HellaSwag (10-shot) | 54.59 |
MMLU (5-shot) | 25.78 |
TruthfulQA (0-shot) | 41.64 |
Winogrande (5-shot) | 56.04 |
GSM8K (5-shot) | 0.45 |
DROP (3-shot) | 5.47 |