|
--- |
|
license: apache-2.0 |
|
tags: |
|
- generated_from_trainer |
|
datasets: |
|
- hendrycks/competition_math |
|
widget: |
|
- text: Find the number of positive divisors of 9!. |
|
example_title: Number theory |
|
- text: Quadrilateral $ABCD$ is a parallelogram. If the measure of angle $A$ is 62 |
|
degrees and the measure of angle $ADB$ is 75 degrees, what is the measure of angle |
|
$ADC$, in degrees? |
|
example_title: Prealgebra |
|
- text: Suppose $x \in [-5,-3]$ and $y \in [2,4]$. What is the largest possible value |
|
of $\frac{x+y}{x-y}$? |
|
example_title: Intermediate algebra |
|
base_model: bert-base-uncased |
|
model-index: |
|
- name: bert-finetuned-math-prob-classification |
|
results: [] |
|
--- |
|
|
|
# bert-finetuned-math-prob-classification |
|
|
|
This model is a fine-tuned version of [bert-base-uncased](https://huggingface.co/bert-base-uncased) on the part of the [competition_math dataset](https://huggingface.co/datasets/competition_math). Specifically, it was trained as a multi-class multi-label model on the problem text. The problem types (labels) used here are "Counting & Probability", "Prealgebra", "Algebra", "Number Theory", "Geometry", "Intermediate Algebra", and "Precalculus". |
|
|
|
## Model description |
|
|
|
See the [bert-base-uncased](https://huggingface.co/bert-base-uncased) model for more details. The only architectural modification made was to the classification head. Here, 7 classes were used. |
|
|
|
## Intended uses & limitations |
|
|
|
This model is intended for demonstration purposes only. The problem type data was in English and contains many LaTeX tokens. |
|
|
|
## Training and evaluation data |
|
|
|
The `problem` field of [competition_math dataset](https://huggingface.co/datasets/competition_math) was used for training and evaluation input data. The target data was taken from the `type` field. |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 5e-05 |
|
- train_batch_size: 8 |
|
- eval_batch_size: 8 |
|
- seed: 42 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- num_epochs: 3.0 |
|
|
|
### Training results |
|
|
|
This fine-tuned model achieves the following result on the problem type competition math test set: |
|
``` |
|
precision recall f1-score support |
|
|
|
Algebra 0.78 0.79 0.79 1187 |
|
Counting & Probability 0.75 0.81 0.78 474 |
|
Geometry 0.76 0.83 0.79 479 |
|
Intermediate Algebra 0.86 0.84 0.85 903 |
|
Number Theory 0.79 0.82 0.80 540 |
|
Prealgebra 0.66 0.61 0.63 871 |
|
Precalculus 0.95 0.89 0.92 546 |
|
|
|
accuracy 0.79 5000 |
|
macro avg 0.79 0.80 0.79 5000 |
|
weighted avg 0.79 0.79 0.79 5000 |
|
``` |
|
|
|
### Framework versions |
|
|
|
- Transformers 4.22.2 |
|
- Pytorch 1.12.1+cu113 |
|
- Datasets 2.5.1 |
|
- Tokenizers 0.12.1 |
|
|
