epoch 40 of 100

README.md

@@ -20,15 +20,15 @@ widget:
 
 **WARNING: Some language produced by this model and README may offend. The model intent is to facilitate bias in AI research**
 
-# notSexistBERT base model (uncased)
+# LessSexistBERT base model (uncased)
 
 Re-pretrained model on English language using a Masked Language Modeling (MLM)
 and Next Sentence Prediction (NSP) objective. It will be introduced in an upcoming
-paper and first released on [HuggingFace](https://huggingface.co/clincolnoz/notSexistBERT_temp). This model is uncased: it does not make a difference between english and English.
+paper and first released on [HuggingFace](https://huggingface.co/clincolnoz/LessSexistBERT). This model is uncased: it does not make a difference between english and English.
 
 ## Model description
 
-sexistBERT is a transformers model pretrained on a **less sexist** corpus of English data in a
+LessSexistBERT is a transformers model pretrained on a **less sexist** corpus of English data in a
 self-supervised fashion. This means it was pretrained on the raw texts only,
 with no humans labeling them in any way (which is why it can use lots of
 publicly available data) with an automatic process to generate inputs and labels
@@ -53,16 +53,16 @@ using the features produced by the BERT model as inputs.
 
 ## Model variations
 
-notSexistBERT has originally been released as sexist and notSexist variations. The uncased models strip out any accent markers.
+LessSexistBERT has originally been released as sexist and notSexist variations. The uncased models strip out any accent markers.
 
 | Model                                                                   | #params   | Language |
 | ----------------------------------------------------------------------- | --------- | -------- |
-| [`sexistBERT`](https://huggingface.co/clincolnoz/sexistBERT_temp)       | 110303292 | English  |
-| [`notSexistBERT`](https://huggingface.co/clincolnoz/notSexistBERT_temp) | 110201784 | English  |
+| [`sexistBERT`](https://huggingface.co/clincolnoz/MoreSexistBERT)       | 110303292 | English  |
+| [`notSexistBERT`](https://huggingface.co/clincolnoz/LessSexistBERT) | 110201784 | English  |
 
 ## Intended uses & limitations
 
-Apart from the usual uses for BERT below, the intended usage of these model is to test bias detection methods and the effect of bias on downstream tasks. SexistBERT is intended to be more biased than notSexistBERT, however that is yet to be determined.
+Apart from the usual uses for BERT below, the intended usage of these model is to test bias detection methods and the effect of bias on downstream tasks. MoreSexistBERT is intended to be more biased than LessSexistBERT, however that is yet to be determined.
 
 You can use the raw model for either masked language modeling or next sentence
 prediction, but it's mostly intended to be fine-tuned on a downstream task. See
@@ -81,29 +81,29 @@ You can use this model directly with a pipeline for masked language modeling:
 
 ```python
 >>> from transformers import pipeline
->>> unmasker = pipeline('fill-mask', model='clincolnoz/notSexistBERT_temp')
+>>> unmasker = pipeline('fill-mask', model='clincolnoz/LessSexistBERT')
 >>> unmasker("Hello I'm a [MASK] model.")
 
-[{'score': 0.5223352313041687,
+[{'score': 0.4557390809059143,
+  'token': 3287,
+  'token_str': 'male',
+  'sequence': "hello i'm a male model."},
+ {'score': 0.10188482701778412,
   'token': 2535,
   'token_str': 'role',
   'sequence': "hello i'm a role model."},
- {'score': 0.12853220105171204,
+ {'score': 0.051661089062690735,
+  'token': 4827,
+  'token_str': 'fashion',
+  'sequence': "hello i'm a fashion model."},
+ {'score': 0.03352942317724228,
+  'token': 18204,
+  'token_str': 'literal',
+  'sequence': "hello i'm a literal model."},
+ {'score': 0.030233129858970642,
   'token': 2449,
   'token_str': 'business',
-  'sequence': "hello i'm a business model."},
- {'score': 0.0621086061000824,
-  'token': 3287,
-  'token_str': 'male',
-  'sequence': "hello i'm a male model."},
- {'score': 0.03042026236653328,
-  'token': 3565,
-  'token_str': 'super',
-  'sequence': "hello i'm a super model."},
- {'score': 0.01949389837682247,
-  'token': 7605,
-  'token_str': '3d',
-  'sequence': "hello i'm a 3d model."}]
+  'sequence': "hello i'm a business model."}]
 ```
 
 Here is how to use this model to get the features of a given text in PyTorch:
@@ -111,12 +111,12 @@ Here is how to use this model to get the features of a given text in PyTorch:
 ```python
 from transformers import BertTokenizer, BertModel
 tokenizer = BertTokenizer.from_pretrained(
-  'clincolnoz/notSexistBERT_temp',
-  revision='v0.34' # tag name, or branch name, or commit hash
+  'clincolnoz/LessSexistBERT',
+  revision='v0.40' # tag name, or branch name, or commit hash
 )
 model = BertModel.from_pretrained(
-  'clincolnoz/notSexistBERT_temp',
-  revision='v0.34' # tag name, or branch name, or commit hash
+  'clincolnoz/LessSexistBERT',
+  revision='v0.40' # tag name, or branch name, or commit hash
 )
 text = "Replace me by any text you'd like."
 encoded_input = tokenizer(text, return_tensors='pt')
@@ -128,13 +128,13 @@ and in TensorFlow:
 ```python
 from transformers import BertTokenizer, TFBertModel
 tokenizer = BertTokenizer.from_pretrained(
-  'clincolnoz/notSexistBERT_temp',
-  revision='v0.34' # tag name, or branch name, or commit hash
+  'clincolnoz/LessSexistBERT',
+  revision='v0.40' # tag name, or branch name, or commit hash
 )
 model = TFBertModel.from_pretrained(
-  'clincolnoz/notSexistBERT_temp',
+  'clincolnoz/LessSexistBERT',
   from_pt=True,
-  revision='v0.34' # tag name, or branch name, or commit hash
+  revision='v0.40' # tag name, or branch name, or commit hash
 )
 text = "Replace me by any text you'd like."
 encoded_input = tokenizer(text, return_tensors='tf')
@@ -148,52 +148,52 @@ neutral, this model can have biased predictions:
 
 ```python
 >>> from transformers import pipeline
->>> unmasker = pipeline('fill-mask', model='clincolnoz/notSexistBERT_temp')
+>>> unmasker = pipeline('fill-mask', model='clincolnoz/LessSexistBERT')
 >>> unmasker("The man worked as a [MASK].")
 
-[{'score': 0.1064024269580841,
-  'token': 5160,
-  'token_str': 'lawyer',
-  'sequence': 'the man worked as a lawyer.'},
- {'score': 0.06261951476335526,
+[{'score': 0.498240202665329,
+  'token': 8872,
+  'token_str': 'cop',
+  'sequence': 'the man worked as a cop.'},
+ {'score': 0.07540689408779144,
+  'token': 15812,
+  'token_str': 'bartender',
+  'sequence': 'the man worked as a bartender.'},
+ {'score': 0.031155399978160858,
+  'token': 17907,
+  'token_str': 'accountant',
+  'sequence': 'the man worked as a accountant.'},
+ {'score': 0.017916174605488777,
+  'token': 6821,
+  'token_str': 'nurse',
+  'sequence': 'the man worked as a nurse.'},
+ {'score': 0.015161702409386635,
   'token': 7155,
   'token_str': 'scientist',
-  'sequence': 'the man worked as a scientist.'},
- {'score': 0.046040475368499756,
-  'token': 10563,
-  'token_str': 'teenager',
-  'sequence': 'the man worked as a teenager.'},
- {'score': 0.04330913722515106,
-  'token': 20273,
-  'token_str': 'programmer',
-  'sequence': 'the man worked as a programmer.'},
- {'score': 0.04167287424206734,
-  'token': 5766,
-  'token_str': 'ceo',
-  'sequence': 'the man worked as a ceo.'}]
+  'sequence': 'the man worked as a scientist.'}]
 
 >>> unmasker("The woman worked as a [MASK].")
 
-[{'score': 0.0949002057313919,
+[{'score': 0.2861696481704712,
+  'token': 8872,
+  'token_str': 'cop',
+  'sequence': 'the woman worked as a cop.'},
+ {'score': 0.20763547718524933,
+  'token': 15812,
+  'token_str': 'bartender',
+  'sequence': 'the woman worked as a bartender.'},
+ {'score': 0.09263389557600021,
+  'token': 15610,
+  'token_str': 'waiter',
+  'sequence': 'the woman worked as a waiter.'},
+ {'score': 0.05527710169553757,
   'token': 6821,
   'token_str': 'nurse',
   'sequence': 'the woman worked as a nurse.'},
- {'score': 0.08425672352313995,
-  'token': 3208,
-  'token_str': 'manager',
-  'sequence': 'the woman worked as a manager.'},
- {'score': 0.07672832906246185,
-  'token': 5160,
-  'token_str': 'lawyer',
-  'sequence': 'the woman worked as a lawyer.'},
- {'score': 0.042527567595243454,
-  'token': 7522,
-  'token_str': 'physician',
-  'sequence': 'the woman worked as a physician.'},
- {'score': 0.034959811717271805,
-  'token': 5766,
-  'token_str': 'ceo',
-  'sequence': 'the woman worked as a ceo.'}]
+ {'score': 0.0525786392390728,
+  'token': 3353,
+  'token_str': 'assistant',
+  'sequence': 'the woman worked as a assistant.'}]
 ```
 
 This bias may also affect all fine-tuned versions of this model.

config.json

@@ -1,5 +1,5 @@
 {
-  "_name_or_path": "bert-base-uncased",
+  "_name_or_path": "/data/cl/notSexistBERT/checkpoint-7871877/",
   "architectures": [
     "BertForPreTraining"
   ],

optimizer.pt

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:64ac355b87c723835fb6e45371d2c0b46cb4e81c0869034b9e97947e4f264142
+oid sha256:f894ca6da2dd8f891a3297a8743baa1afa9d45036a92dd405c70f01f5da8a5e0
 size 881735429

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8e1f83726c3328470db6fe075764c76e5fbaa13081c02f771168748a2603abd1
+oid sha256:9fdcb834e1af55bdf59b05eba508a0483254377aaafeeff7023c50f43f2aacc0
 size 440881865

rng_state.pth

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:60d91c35f1b740f8e9a69153126d31e72877822aef7f579faf648f3b6fb77a4d
+oid sha256:ed6a0029ec1778333f97ab2acb8cdd9cf0c47125a9aa99164f25604ba4df502d
 size 14575

scaler.pt

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:76c7e1f094d307d632e596d819670442fc9332e781f5644895c3bea4967aec96
+oid sha256:d57f3b9d531afdda4dabb7ed0be6f19768996a9658b591241cdcf5ccacd40f38
 size 557

scheduler.pt

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6bfdf5e528533d6b015bfcb7d44cd7015582f4bf445d6bfd01bd740bbe7abd1b
+oid sha256:365b0fe5e8f1e15692ec22a488e73e87d7700434fe9540e7a2d5cb07d7c35ae7
 size 627

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:122a872cb609b5fc05adf84a8d6d9565266277c61206274491cd92c2a443a99f
+oid sha256:1cfc6b60d9b6d24dbaf9d97d9365ac2ddaf991ef1860605a7cfa32d631957c38
 size 3515