JuliusFx/dyu-fr-t5-small_v7

This model is a fine-tuned version of t5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Train Loss: 1.9053
• Validation Loss: 3.0844
• Epoch: 99

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• optimizer: {'name': 'AdamWeightDecay', 'learning_rate': 2e-05, 'decay': 0.0, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False, 'weight_decay_rate': 0.01}
• training_precision: float32

Training results

Train Loss	Validation Loss	Epoch
3.6620	3.3803	0
3.4472	3.2921	1
3.3525	3.2231	2
3.2837	3.1862	3
3.2322	3.1474	4
3.1837	3.1283	5
3.1428	3.0978	6
3.1095	3.0848	7
3.0765	3.0664	8
3.0453	3.0565	9
3.0144	3.0408	10
2.9884	3.0344	11
2.9633	3.0285	12
2.9377	3.0228	13
2.9175	3.0158	14
2.8979	3.0310	15
2.8737	3.0306	16
2.8575	3.0122	17
2.8343	3.0232	18
2.8178	3.0135	19
2.7992	3.0038	20
2.7791	3.0221	21
2.7636	3.0123	22
2.7430	3.0083	23
2.7286	3.0186	24
2.7083	2.9942	25
2.6964	2.9911	26
2.6792	2.9891	27
2.6580	3.0056	28
2.6414	3.0048	29
2.6329	3.0040	30
2.6213	3.0035	31
2.6042	3.0061	32
2.5913	3.0095	33
2.5720	3.0202	34
2.5590	3.0204	35
2.5429	3.0304	36
2.5352	3.0128	37
2.5162	2.9989	38
2.5086	3.0094	39
2.4949	3.0048	40
2.4799	3.0187	41
2.4703	3.0199	42
2.4537	3.0340	43
2.4468	3.0233	44
2.4317	3.0171	45
2.4195	3.0274	46
2.4079	3.0265	47
2.3948	3.0173	48
2.3852	3.0194	49
2.3728	3.0275	50
2.3631	3.0147	51
2.3525	3.0338	52
2.3401	3.0444	53
2.3303	3.0556	54
2.3145	3.0440	55
2.3057	3.0500	56
2.2951	3.0496	57
2.2830	3.0497	58
2.2690	3.0461	59
2.2646	3.0373	60
2.2503	3.0343	61
2.2457	3.0589	62
2.2343	3.0538	63
2.2285	3.0434	64
2.2146	3.0410	65
2.2048	3.0339	66
2.1913	3.0507	67
2.1803	3.0459	68
2.1747	3.0487	69
2.1641	3.0344	70
2.1547	3.0440	71
2.1461	3.0655	72
2.1403	3.0383	73
2.1267	3.0239	74
2.1161	3.0183	75
2.1010	3.0555	76
2.0980	3.0412	77
2.0894	3.0400	78
2.0806	3.0389	79
2.0744	3.0377	80
2.0591	3.0596	81
2.0525	3.0449	82
2.0465	3.0532	83
2.0385	3.0465	84
2.0232	3.0374	85
2.0231	3.0280	86
2.0089	3.0506	87
2.0031	3.0629	88
1.9959	3.0440	89
1.9854	3.0669	90
1.9776	3.0718	91
1.9698	3.0657	92
1.9591	3.0650	93
1.9529	3.0599	94
1.9483	3.0726	95
1.9429	3.0682	96
1.9271	3.0618	97
1.9208	3.0857	98
1.9053	3.0844	99

Framework versions

• Transformers 4.38.2
• TensorFlow 2.15.0
• Datasets 2.18.0
• Tokenizers 0.15.2