fine_tuned_bart

This model is a fine-tuned version of facebook/bart-large-cnn on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.7075

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	8	0.7205
0.4729	2.0	16	0.7109
0.4794	3.0	24	0.7048
0.4716	4.0	32	0.7081
0.476	5.0	40	0.7095
0.476	6.0	48	0.7174
0.4751	7.0	56	0.7050
0.4683	8.0	64	0.7047
0.4583	9.0	72	0.7058
0.474	10.0	80	0.7045
0.474	11.0	88	0.7062
0.4651	12.0	96	0.7047
0.4523	13.0	104	0.7028
0.4626	14.0	112	0.7049
0.4634	15.0	120	0.7067
0.4634	16.0	128	0.7091
0.4543	17.0	136	0.7087
0.4502	18.0	144	0.7084
0.4604	19.0	152	0.7098
0.4503	20.0	160	0.7065
0.4503	21.0	168	0.7046
0.4642	22.0	176	0.7033
0.4334	23.0	184	0.7029
0.4626	24.0	192	0.7037
0.4584	25.0	200	0.7046
0.4584	26.0	208	0.7063
0.4508	27.0	216	0.7075
0.4498	28.0	224	0.7078
0.4532	29.0	232	0.7077
0.4514	30.0	240	0.7075

Framework versions

• PEFT 0.12.0
• Transformers 4.42.4
• Pytorch 2.3.1+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1