Hugging Face's logo

tmnam
/
vihealthbert-w_unsup-SynPD

Fill-Mask
Transformers
Safetensors
roberta
Generated from Trainer
Eval Results
Inference Endpoints
Model card Files Community
tmnam20 commited on
Commit
cc2cfb0
•
1 Parent(s): e1fb2df

Model save

Browse files
Files changed (2) hide show
  1. README.md +35 -30
  2. model.safetensors +1 -1
README.md CHANGED
@@ -2,24 +2,11 @@
2
  base_model: demdecuong/vihealthbert-base-word
3
  tags:
4
  - generated_from_trainer
5
- datasets:
6
- - tmnam20/pretrained-vn-med-nli
7
  metrics:
8
  - accuracy
9
  model-index:
10
  - name: vihealthbert-w_unsup-SynPD
11
- results:
12
- - task:
13
- name: Masked Language Modeling
14
- type: fill-mask
15
- dataset:
16
- name: tmnam20/pretrained-vn-med-nli all
17
- type: tmnam20/pretrained-vn-med-nli
18
- args: all
19
- metrics:
20
- - name: Accuracy
21
- type: accuracy
22
- value: 0.686153705209395
23
  ---
24
 
25
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -27,10 +14,10 @@ should probably proofread and complete it, then remove this comment. -->
27
 
28
  # vihealthbert-w_unsup-SynPD
29
 
30
- This model is a fine-tuned version of [demdecuong/vihealthbert-base-word](https://huggingface.co/demdecuong/vihealthbert-base-word) on the tmnam20/pretrained-vn-med-nli all dataset.
31
  It achieves the following results on the evaluation set:
32
- - Loss: 1.5768
33
- - Accuracy: 0.6862
34
 
35
  ## Model description
36
 
@@ -52,7 +39,7 @@ The following hyperparameters were used during training:
52
  - learning_rate: 3e-05
53
  - train_batch_size: 32
54
  - eval_batch_size: 16
55
- - seed: 21363
56
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
57
  - lr_scheduler_type: linear
58
  - lr_scheduler_warmup_ratio: 0.1
@@ -62,22 +49,40 @@ The following hyperparameters were used during training:
62
 
63
  | Training Loss | Epoch | Step | Validation Loss | Accuracy |
64
  |:-------------:|:------:|:-----:|:---------------:|:--------:|
65
- | 7.0234 | 0.8616 | 5000 | 2.5909 | 0.5576 |
66
- | 5.2736 | 1.7232 | 10000 | 2.1890 | 0.5962 |
67
- | 4.9126 | 2.5849 | 15000 | 1.9095 | 0.6381 |
68
- | 4.791 | 3.4465 | 20000 | 1.8286 | 0.6469 |
69
- | 4.6538 | 4.3081 | 25000 | 1.7144 | 0.6644 |
70
- | 4.5846 | 5.1697 | 30000 | 1.6779 | 0.6704 |
71
- | 4.5568 | 6.0314 | 35000 | 1.6362 | 0.6766 |
72
- | 4.5079 | 6.8930 | 40000 | 1.6008 | 0.6814 |
73
- | 4.469 | 7.7546 | 45000 | 1.6064 | 0.6805 |
74
- | 4.4514 | 8.6162 | 50000 | 1.5800 | 0.6852 |
75
- | 4.4317 | 9.4779 | 55000 | 1.5540 | 0.6880 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
76
 
77
 
78
  ### Framework versions
79
 
80
  - Transformers 4.40.2
81
- - Pytorch 2.0.1+cu118
82
  - Datasets 2.19.1
83
  - Tokenizers 0.19.1
 
2
  base_model: demdecuong/vihealthbert-base-word
3
  tags:
4
  - generated_from_trainer
 
 
5
  metrics:
6
  - accuracy
7
  model-index:
8
  - name: vihealthbert-w_unsup-SynPD
9
+ results: []
 
 
 
 
 
 
 
 
 
 
 
10
  ---
11
 
12
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 
14
 
15
  # vihealthbert-w_unsup-SynPD
16
 
17
+ This model is a fine-tuned version of [demdecuong/vihealthbert-base-word](https://huggingface.co/demdecuong/vihealthbert-base-word) on an unknown dataset.
18
  It achieves the following results on the evaluation set:
19
+ - Loss: 1.5576
20
+ - Accuracy: 0.6883
21
 
22
  ## Model description
23
 
 
39
  - learning_rate: 3e-05
40
  - train_batch_size: 32
41
  - eval_batch_size: 16
42
+ - seed: 19144
43
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
44
  - lr_scheduler_type: linear
45
  - lr_scheduler_warmup_ratio: 0.1
 
49
 
50
  | Training Loss | Epoch | Step | Validation Loss | Accuracy |
51
  |:-------------:|:------:|:-----:|:---------------:|:--------:|
52
+ | 5.8543 | 0.3446 | 2000 | 3.8967 | 0.3950 |
53
+ | 3.4544 | 0.6893 | 4000 | 2.8119 | 0.5306 |
54
+ | 2.8312 | 1.0339 | 6000 | 2.4040 | 0.5771 |
55
+ | 2.5914 | 1.3786 | 8000 | 2.6482 | 0.5350 |
56
+ | 2.5649 | 1.7232 | 10000 | 2.1335 | 0.6087 |
57
+ | 2.2749 | 2.0679 | 12000 | 1.9895 | 0.6282 |
58
+ | 2.1572 | 2.4125 | 14000 | 1.9313 | 0.6353 |
59
+ | 2.1009 | 2.7572 | 16000 | 1.8646 | 0.6429 |
60
+ | 2.0609 | 3.1018 | 18000 | 1.8572 | 0.6450 |
61
+ | 2.0885 | 3.4465 | 20000 | 1.9489 | 0.6285 |
62
+ | 1.9891 | 3.7911 | 22000 | 1.7700 | 0.6583 |
63
+ | 1.9368 | 4.1358 | 24000 | 1.7398 | 0.6609 |
64
+ | 1.9003 | 4.4804 | 26000 | 1.7165 | 0.6664 |
65
+ | 1.9058 | 4.8251 | 28000 | 1.7032 | 0.6670 |
66
+ | 1.859 | 5.1697 | 30000 | 1.6771 | 0.6718 |
67
+ | 1.8401 | 5.5144 | 32000 | 1.6652 | 0.6710 |
68
+ | 1.8181 | 5.8590 | 34000 | 1.6417 | 0.6754 |
69
+ | 1.8133 | 6.2037 | 36000 | 1.6431 | 0.6748 |
70
+ | 1.7888 | 6.5483 | 38000 | 1.6363 | 0.6755 |
71
+ | 1.7811 | 6.8930 | 40000 | 1.6205 | 0.6793 |
72
+ | 1.7481 | 7.2376 | 42000 | 1.6190 | 0.6807 |
73
+ | 1.7509 | 7.5823 | 44000 | 1.6142 | 0.6794 |
74
+ | 1.7517 | 7.9269 | 46000 | 1.5949 | 0.6819 |
75
+ | 1.7358 | 8.2716 | 48000 | 1.5909 | 0.6843 |
76
+ | 1.7287 | 8.6162 | 50000 | 1.5757 | 0.6851 |
77
+ | 1.7132 | 8.9609 | 52000 | 1.5671 | 0.6885 |
78
+ | 1.7042 | 9.3055 | 54000 | 1.5685 | 0.6867 |
79
+ | 1.7051 | 9.6502 | 56000 | 1.5609 | 0.6876 |
80
+ | 1.7051 | 9.9948 | 58000 | 1.5576 | 0.6883 |
81
 
82
 
83
  ### Framework versions
84
 
85
  - Transformers 4.40.2
86
+ - Pytorch 2.2.0+cu121
87
  - Datasets 2.19.1
88
  - Tokenizers 0.19.1
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:2a7226f661dd3a934caf37dd924e34d250d9d45134c827392f61b1f7d8069417
3
  size 540279452
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a1b789c4c8bb7da3be52210541d04d9e938c01f03c9f44e4ac8beaabb3bdf40f
3
  size 540279452