bert-base-uncased-finetuned-glue-sst2
Use for sentiment analysis.
This model is a fine-tuned version of bert-base-uncased on the glue sst2 dataset. The model achieves 91.39%
accuracy on the validation dataset.
Model description
bert-base-uncased is a pretrained English language model. bert-base-uncased-finetuned-glue-sst2
adds a 2-class classification head for predicting positive
and negative
sentiment.
Training and evaluation data
The model has been trained on 10K training samples, even though the glue sst2 dataset contains 67.3K samples. This was done to decrease training time.
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': {'module': 'keras.optimizers.schedules', 'class_name': 'PolynomialDecay', 'config': {'initial_learning_rate': 5e-05, 'decay_steps': 3750, 'end_learning_rate': 0.0, 'power': 1.0, 'cycle': False, 'name': None}, 'registered_name': None}, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
- training_precision: float32
Training results
- Accuracy (training):
94.33%
- Accuracy (validation):
91.39%
Framework versions
- Transformers 4.35.2
- TensorFlow 2.15.0
- Datasets 2.16.1
- Tokenizers 0.15.0
- Downloads last month
- 14
Inference API (serverless) is not available, repository is disabled.
Model tree for raj-p/bert-base-uncased-finetuned-glue-sst2
Base model
google-bert/bert-base-uncased
Finetuned
this model