metadata
language:
- en
license: apache-2.0
library_name: sentence-transformers
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:900
- loss:MatryoshkaLoss
- loss:MultipleNegativesRankingLoss
base_model: BAAI/bge-base-en-v1.5
datasets: []
metrics:
- cosine_accuracy@1
- cosine_accuracy@3
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@3
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@3
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@10
- cosine_mrr@10
- cosine_map@100
widget:
- source_sentence: >-
ographics, analyst reports and more.
Knowledge Center
Learn about the data privacy, security and governance landscape.
Securiti Education
Courses and Certifications for data privacy, security and governance
professionals.
Company
About Us
Learn all about Securiti, our mission and history
Partner Program
Join our Partner Program
Contact Us
Contact us to learn more or schedule a demo
News Coverage
Read about Securiti in the news
Press Releases
Find our latest press releases
Careers
Join the talented Securiti team
Knowledge Center » Data Privacy Automation
By Securiti Research Team
Published February 2, 2021 / Updated September 28, 2023
The Irish Data Protection Act, 2018 (Irish DPA) implements the General
Data Protection Regulation (GDPR) and transposes the European Union Law
Enforcement Directive in Ireland. Since it incorporates most of the
provisions from the GDPR and the Law Enforcement Directive with limited
additions and deletions as per the national law, it is considered to be
the principal data protection legislation in Ireland.
Table of contents
Rights of Data Subjects
Responsibilities of data controllers
Irish DPC Cookie Consent Guidelines
Automating Compliance
The Irish DPA provides the same rights to data subjects with respect to
their personal data as that of the GDPR. These rights give data subjects
control over their data and may be processed under particular conditions
and limitations.
Data subjects have the right to be informed of when and how their data is
being used and collected. This refers to the obligation of the data
controller to inform and notify any relevant details to the data subjects
for any important action taken on their data.
On a request of the data subject, an organization must provide data
subject access to his/her personal data and information about the ways
personal data has been or may have been used, disclosed, or processed by
the organization.
This right applies when the accuracy of data is contested by the data
subject and when processing is unlawful and the data subject opposes the
deletion of the data. Data subjects need to be informed before any such
restriction is lifted.
sentences:
- >-
What is the PDPA Act in Malaysia and how does it regulate the processing
of personal data?
- What is the purpose of the European Union GDPR?
- >-
What does the Irish Data Protection Act of 2018 implement in relation to
the Law Enforcement Directive?
- source_sentence: >-
Learn all about Securiti, our mission and history
Partner Program
Join our Partner Program
Contact Us
Contact us to learn more or schedule a demo
News Coverage
Read about Securiti in the news
Press Releases
Find our latest press releases
Careers
Join the talented Securiti team
Blog » Data Consent Automation
By Securiti Research Team
Published October 1, 2020 / Updated October 6, 2023
On 6 April, the Data Protection Commission of Ireland (DPC) released a
substantive Guidance Note on cookies (Guidance) and provided organizations
a grace period of six months to ensure compliance. After the end of the
six- month window, which is 5 October 2020, the Irish DPC may act to
enforce the Guidance and can hold organizations liable for failing to
obtain valid consent before the processing of cookies.
This Guidance was issued based on the report released by the DPC on the
findings of a “cookie sweep survey”. The survey was conducted on around 38
organizations operating within the territory of Ireland and around 35 of
those companies were found to be significantly lacking in cookie
compliance requirements. The DPC noticed the following non-compliance
practices of organizations, among others:
Dropping of non
essential cookies on landing pages without obtaining user’s consent,
The lifespans of most cookies that are dropped are not proportionate to
the purposes of the cookies,
Inadequate cookie banners,
Frequent use of pre
checked boxes for the processing of non
essential cookies,
A lack of stand
alone cookie policies,
Failure to fulfill the requirements of a valid consent as per the General
Data Protection Regulation (GDPR) and the Irish e
Privacy Regulations.
Based on its identification of the above non-compliance areas, the Irish
DPC released the comprehensive Guidance for organizations. The Guidance
explains the purposes of cookies as well as it adheres to the requirements
of the GDPR, e-Privacy Directive, and the Guidelines on Consent of the
European Data Protection Board, released on 4 May 2020 that declared
cookie walls invalid.
_Read EDPB’s Updated Guidelines on Consent_
The Guidance also complements the landmark decision by the Court of
Justice of the
sentences:
- >-
What are the requirements for valid consent under the GDPR and Irish
e-Privacy Regulations according to the Irish DPC's Guidance on cookies?
- What are the CPPA's duties in enforcing CCPA and CPRA?
- >-
What legislative measures has Spain taken to protect citizens' personal
information and data, and how does it compare to Saudi Arabia's data
protection law?
- source_sentence: >2-
are:
Regulation No. 20 of 2016 concerning Protection of Personal Data in
Electronic Systems (MoCI Reg);
Amended Law No. 11 of 2008 on Electronic Information and Transaction (EIT
Law);
Government Regulation No. 71 of 2019 on the Implementation of the
Electronic System and Transaction (GR 71).
There are also sectoral regulations that regulate the personal data in a
specific sector e.g, banking sector, health sector, etc.
Following delays due to COVID-19, Indonesia is now geared to pass its
first Personal Data Protection Act (PDP Law). On January 24, 2020, the
bill’s final draft was submitted to the Indonesian House of
Representatives. The PDP law will address the much-needed reforms to the
country’s data privacy protection rules. The law is built on the European
Union’s General Data Protection Regulation (GDPR).
In essence, Indonesia will soon follow the same data subject rights and
personal data processing regulations set by the European Union in their
GDPR.
The PDP Law will have 72 articles across 15 chapters. These articles and
chapters will extensively cover data ownership rights, prohibitions on
data use, along with the collection, storage, processing, and transfer of
personal data of Indonesian users.
With Indonesia being an active part of the global economy and attracting
millions of tourists annually, businesses should quickly align their
business operations to comply with the upcoming PDP law.
The PDP law will impact local businesses in Indonesia and will also have
an impact on companies across the globe that deal with Indonesian
consumers. . The PDP law will apply to any registered company dealing with
Indonesian residents, irrespective of where they are registered.
Whether an entity is public or private, local or international, the PDP
Law will automatically apply to them if they deal with the personal data
of Indonesian residents. The new PDP Law is expected to apply to all
sectors, bringing forward comprehensive provisions on personal data
protection, both electronically and non-electronically.
The PDP law will regulate sensitive personal data as well as other
personal data that may endanger or harm the privacy of the data subject.
The PDP Bill applies to companies both within and outside of the territory
of Indonesia where
sentences:
- >-
What does Securiti offer businesses in terms of automating privacy and
security processes, and why is it important for businesses to embrace
robotic automation for compliance?
- >-
What companies will be impacted by Indonesia's Personal Data Protection
Law?
- >-
What is the purpose of the Data Command Center in relation to the
company's products and solutions?
- source_sentence: >-
Contact us to learn more or schedule a demo
News Coverage
Read about Securiti in the news
Press Releases
Find our latest press releases
Careers
Join the talented Securiti team
Knowledge Center » Data Privacy Automation
By Securiti Research Team
Published August 9, 2021 / Updated October 2, 2023
In China, the following are three main laws that cover the data privacy
and data security regime:
The Cybersecurity Law of the People’s Republic of China (the “CSL”),
implemented on June 1, 2017.
The Personal Information Protection Law of the People’s Republic of China
(the “PIPL”), effective from November 1, 2021.
The Data Security Law (the “DSL”), will be implemented from September 1,
2021.
The focus of this article is on the DSL that was promulgated to
standardize data processing activities, ensure data security, promote data
development and utilization, and protect the legitimate rights and
interests of individuals and organizations.
Table of contents
Scope of Application and Extraterritorial Effect of DSL
Penalties for Non
Compliance
How Securiti Can Help
The DSL applies to and regulates data processing activities by
organizations and individuals, and security supervision of such activities
within the territory of China. The DSL also regulates data processing
activities conducted outside of China that harm China’s national security
or the public interest, or the legal interests of citizens and
organizations in China. It would be right to state that DSL has extensive
and extra-territorial application. It imposes a number of obligations on
organizations and individuals even those that are not based in China
regarding data categorization and classification, data risk controls and
risk assessments, cross-border data transfers, and data export controls.
The DSL applies to data recorded in electronic and other forms including
digital and cyber information, and information recorded in other forms
such as paper records. Data processing activities regulated by DSL
include, without limitation, the collection, storage, use, processing,
transmission, provision, or disclosure of data.
Organizations and individuals need to understand and fulfill the following
requirements of the DSL in order to avoid unnecessary compliance risks and
penalties:
sentences:
- >-
What can organizations do to manage vendor risk in relation to data
protection and compliance, considering Kenya's Data Protection Act 2019?
- >-
What are the time limits for organizations to respond to a request under
the NZ Privacy Act 2020, and what are the requirements for transferring
personal information outside NZ?
- >-
How does the Data Security Law in China contribute to data
standardization, security protection, and development?
- source_sentence: >-
Oblige with Data Localization Requirements:
Oblige with Product Safety and Certifications Requirements:
Fulfill Content Monitoring Requirements:
China’s Cybersecurity Law (the “CSL”), which went into effect on June 1st,
2017, applies to the construction, operation, maintenance, and use of
information networks, and the supervision and administration of
cybersecurity in China. The CSL provides guidelines on cybersecurity
requirements for safeguarding Chinese cyberspace. The law protects the
legal interests and rights of organizations as well as individuals in
China. It also promotes the secure development of technology and the
digitization of the economy in China. Following entities come under the
application scope of the CSL:
**Network Operators:
** It refers to the owners and administrators of networks and network
service providers, and could be interpreted to include any companies
providing services, or running their business through a computer network
in China.
**Critical Information Infrastructure Operators (CIIOs):
** It refers to operators of critical information infrastructure in
important industries and sectors (such as information service, public
service, and e
government) and other information infrastructure that, if leaked, may
severely threaten the national security, national economy, people’s
livelihood, and public interests.
**Network Products and Services Providers:
** Organizations that provide information through networks or provide
services to obtain information, including users, network services
providers which provide network tools, devices, media, etc.
Compliance with the CSL is not straightforward since CSL has several
ambiguities and complicated obligations for network operators and CIIOs.
Additional laws and guidelines will also be considered concerning the CSL
compliance, including guidelines concerning the security assessment of
cross- border transfers of personal information and important data, Data
Security Law (DSL), and recently promulgated Personal Information
Protection Law (PIPL).
We have prepared the following compliance checklist for the covered
entities to ensure compliance with the CSL. Please note that this is not
an exhaustive compliance list. For a detailed overview of the CSL, please
refer to our article on What is China’s Cybersecurity Law?
Network operators must adopt the following security measures to prevent
network interference, damage, or unauthorized access, and prevent network
data from leakage, theft, or alteration:
Establish internal,
Cybersecurity product manufacturers, security service suppliers, and other
organizations that provide services through networks should oblige with
the following requirements:
Network products and services providers must not set up malicious
programs.
Upon discovering a security flaw, vulnerability, or another risk in their
product or service, they must take remedial action immediately, inform
users and report the issue to the relevant departments.
Network product and service providers are required to conduct security
maintenance for their products and services.
CIIOs must, when procuring network products and services that may impact
national security, submit the products and services to CAC and the State
Council departments for a review for national security purposes. Critical
network equipment and special cybersecurity products can only be sold or
provided after being certified by a qualified establishment, and are in
compliance with national standards.
According to Article 47 of the CSL, network operators are required to
monitor the information released by their users for information that is
“prohibited from being published or transmitted by laws or administrative
regulations. If such information is discovered, network operators must
cease the transmission of information, remove the information, keep
records, and report any unlawful content to relevant authorities. Securiti
helps organizations automate their privacy management operations using
artificial intelligence and robotic automation. Request a demo and start
your CSL compliance process today.
Get all the latest information, law updates and more delivered to your
inbox
Copy
55
View More
September 11, 2023
Compliance Software
Securiti has just been recognized as a Leader in the “IDC MarketScape:
Worldwide Data Privacy Compliance Software 2023 Vendor Assessment” report.
This makes us...
View More
May 10, 2023
by
Design and Privacy
by
Default
Privacy-by-design and privacy-by-default are two cornerstone concepts of
data protection regulatory frameworks. Thus, compliance thereof is an
essential legal prerequisite for any entity which...
View More
April 5,
sentences:
- >-
What are the 13 IPPs of New Zealand's Privacy Act 2020 that apply to all
organizations, including those outside of New Zealand, offering
goods/services to individuals in New Zealand or collecting information
about individuals in New Zealand?
- >-
What security measures must network operators adopt to fulfill content
monitoring requirements under China's Cybersecurity Law, and what
obligations do network products and services providers and CIIOs have in
relation to product safety and certifications?
- >-
How does the PDPA in Malaysia protect personal data in commercial
transactions and who does it apply to?
pipeline_tag: sentence-similarity
model-index:
- name: SentenceTransformer based on BAAI/bge-base-en-v1.5
results:
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 768
type: dim_768
metrics:
- type: cosine_accuracy@1
value: 0.08
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.27
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.45
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.67
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.08
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.09
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.08999999999999998
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.06699999999999999
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.08
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.27
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.45
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.67
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.33828063637415534
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.23589682539682535
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.24406326043435023
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 512
type: dim_512
metrics:
- type: cosine_accuracy@1
value: 0.06
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.25
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.39
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.66
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.06
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.08333333333333331
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.07800000000000001
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.06599999999999999
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.06
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.25
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.39
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.66
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.31695711820935435
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.2123928571428571
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.22150012925090945
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 256
type: dim_256
metrics:
- type: cosine_accuracy@1
value: 0.05
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.24
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.38
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.6
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.05
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.08
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.07600000000000001
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.05999999999999999
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.05
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.24
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.38
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.6
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.2931065726305541
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.19853174603174606
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.21132630111968292
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 128
type: dim_128
metrics:
- type: cosine_accuracy@1
value: 0.05
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.28
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.36
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.55
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.05
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.09333333333333334
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.07200000000000001
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.05499999999999999
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.05
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.28
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.36
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.55
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.278284909333787
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.19384126984126987
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.20776022518923803
name: Cosine Map@100
- task:
type: information-retrieval
name: Information Retrieval
dataset:
name: dim 64
type: dim_64
metrics:
- type: cosine_accuracy@1
value: 0.04
name: Cosine Accuracy@1
- type: cosine_accuracy@3
value: 0.21
name: Cosine Accuracy@3
- type: cosine_accuracy@5
value: 0.3
name: Cosine Accuracy@5
- type: cosine_accuracy@10
value: 0.53
name: Cosine Accuracy@10
- type: cosine_precision@1
value: 0.04
name: Cosine Precision@1
- type: cosine_precision@3
value: 0.07
name: Cosine Precision@3
- type: cosine_precision@5
value: 0.06000000000000001
name: Cosine Precision@5
- type: cosine_precision@10
value: 0.05299999999999999
name: Cosine Precision@10
- type: cosine_recall@1
value: 0.04
name: Cosine Recall@1
- type: cosine_recall@3
value: 0.21
name: Cosine Recall@3
- type: cosine_recall@5
value: 0.3
name: Cosine Recall@5
- type: cosine_recall@10
value: 0.53
name: Cosine Recall@10
- type: cosine_ndcg@10
value: 0.25162020276083924
name: Cosine Ndcg@10
- type: cosine_mrr@10
value: 0.16670634920634916
name: Cosine Mrr@10
- type: cosine_map@100
value: 0.177182977653562
name: Cosine Map@100
SentenceTransformer based on BAAI/bge-base-en-v1.5
This is a sentence-transformers model finetuned from BAAI/bge-base-en-v1.5. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
Model Details
Model Description
- Model Type: Sentence Transformer
- Base model: BAAI/bge-base-en-v1.5
- Maximum Sequence Length: 512 tokens
- Output Dimensionality: 768 tokens
- Similarity Function: Cosine Similarity
- Language: en
- License: apache-2.0
Model Sources
Full Model Architecture
SentenceTransformer(
(0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel
(1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
(2): Normalize()
)
Usage
Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
pip install -U sentence-transformers
Then you can load this model and run inference.
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("MugheesAwan11/bge-base-securiti-dataset-1-v7")
sentences = [
'\n\nOblige with Data Localization Requirements:\n\nOblige with Product Safety and Certifications Requirements:\n\nFulfill Content Monitoring Requirements:\n\nChina’s Cybersecurity Law (the “CSL”), which went into effect on June 1st, 2017, applies to the construction, operation, maintenance, and use of information networks, and the supervision and administration of cybersecurity in China. The CSL provides guidelines on cybersecurity requirements for safeguarding Chinese cyberspace. The law protects the legal interests and rights of organizations as well as individuals in China. It also promotes the secure development of technology and the digitization of the economy in China. Following entities come under the application scope of the CSL:\n\n**Network Operators:\n\n** It refers to the owners and administrators of networks and network service providers, and could be interpreted to include any companies providing services, or running their business through a computer network in China.\n\n**Critical Information Infrastructure Operators (CIIOs):\n\n** It refers to operators of critical information infrastructure in important industries and sectors (such as information service, public service, and e\n\ngovernment) and other information infrastructure that, if leaked, may severely threaten the national security, national economy, people’s livelihood, and public interests.\n\n**Network Products and Services Providers:\n\n** Organizations that provide information through networks or provide services to obtain information, including users, network services providers which provide network tools, devices, media, etc.\n\nCompliance with the CSL is not straightforward since CSL has several ambiguities and complicated obligations for network operators and CIIOs. Additional laws and guidelines will also be considered concerning the CSL compliance, including guidelines concerning the security assessment of cross- border transfers of personal information and important data, Data Security Law (DSL), and recently promulgated Personal Information Protection Law (PIPL).\n\nWe have prepared the following compliance checklist for the covered entities to ensure compliance with the CSL. Please note that this is not an exhaustive compliance list. For a detailed overview of the CSL, please refer to our article on What is China’s Cybersecurity Law?\n\n## 1\\. Fulfill Network Operations Security Requirements:\n\n## A. Requirements for network operators:\n\nNetwork operators must adopt the following security measures to prevent network interference, damage, or unauthorized access, and prevent network data from leakage, theft, or alteration:\n\nEstablish internal, \n## 5\\. Oblige with Product Safety and Certifications Requirements:\n\n## A. Requirements for Network Products and Services Providers:\n\nCybersecurity product manufacturers, security service suppliers, and other organizations that provide services through networks should oblige with the following requirements:\n\nNetwork products and services providers must not set up malicious programs.\n\nUpon discovering a security flaw, vulnerability, or another risk in their product or service, they must take remedial action immediately, inform users and report the issue to the relevant departments.\n\nNetwork product and service providers are required to conduct security maintenance for their products and services.\n\n## B. Requirements for CIIOs:\n\nCIIOs must, when procuring network products and services that may impact national security, submit the products and services to CAC and the State Council departments for a review for national security purposes. Critical network equipment and special cybersecurity products can only be sold or provided after being certified by a qualified establishment, and are in compliance with national standards.\n\n## 6\\. Fulfill Content Monitoring Requirements:\n\nAccording to Article 47 of the CSL, network operators are required to monitor the information released by their users for information that is “prohibited from being published or transmitted by laws or administrative regulations. If such information is discovered, network operators must cease the transmission of information, remove the information, keep records, and report any unlawful content to relevant authorities. Securiti helps organizations automate their privacy management operations using artificial intelligence and robotic automation. Request a demo and start your CSL compliance process today.\n\n## Join Our Newsletter\n\nGet all the latest information, law updates and more delivered to your inbox\n\n### Share\n\nCopy\n\n55\n\n### More Stories that May Interest You\n\nView More\n\nSeptember 11, 2023\n\n## Securiti named a Leader in the IDC MarketScape for Data Privacy Compliance Software\n\nSecuriti has just been recognized as a Leader in the “IDC MarketScape: Worldwide Data Privacy Compliance Software 2023 Vendor Assessment” report. This makes us...\n\nView More\n\nMay 10, 2023\n\n## Privacy\n\nby\n\nDesign and Privacy\n\nby\n\nDefault\n\nPrivacy-by-design and privacy-by-default are two cornerstone concepts of data protection regulatory frameworks. Thus, compliance thereof is an essential legal prerequisite for any entity which...\n\nView More\n\nApril 5,',
"What security measures must network operators adopt to fulfill content monitoring requirements under China's Cybersecurity Law, and what obligations do network products and services providers and CIIOs have in relation to product safety and certifications?",
'How does the PDPA in Malaysia protect personal data in commercial transactions and who does it apply to?',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
Evaluation
Metrics
Information Retrieval
| Metric |
Value |
| cosine_accuracy@1 |
0.08 |
| cosine_accuracy@3 |
0.27 |
| cosine_accuracy@5 |
0.45 |
| cosine_accuracy@10 |
0.67 |
| cosine_precision@1 |
0.08 |
| cosine_precision@3 |
0.09 |
| cosine_precision@5 |
0.09 |
| cosine_precision@10 |
0.067 |
| cosine_recall@1 |
0.08 |
| cosine_recall@3 |
0.27 |
| cosine_recall@5 |
0.45 |
| cosine_recall@10 |
0.67 |
| cosine_ndcg@10 |
0.3383 |
| cosine_mrr@10 |
0.2359 |
| cosine_map@100 |
0.2441 |
Information Retrieval
| Metric |
Value |
| cosine_accuracy@1 |
0.06 |
| cosine_accuracy@3 |
0.25 |
| cosine_accuracy@5 |
0.39 |
| cosine_accuracy@10 |
0.66 |
| cosine_precision@1 |
0.06 |
| cosine_precision@3 |
0.0833 |
| cosine_precision@5 |
0.078 |
| cosine_precision@10 |
0.066 |
| cosine_recall@1 |
0.06 |
| cosine_recall@3 |
0.25 |
| cosine_recall@5 |
0.39 |
| cosine_recall@10 |
0.66 |
| cosine_ndcg@10 |
0.317 |
| cosine_mrr@10 |
0.2124 |
| cosine_map@100 |
0.2215 |
Information Retrieval
| Metric |
Value |
| cosine_accuracy@1 |
0.05 |
| cosine_accuracy@3 |
0.24 |
| cosine_accuracy@5 |
0.38 |
| cosine_accuracy@10 |
0.6 |
| cosine_precision@1 |
0.05 |
| cosine_precision@3 |
0.08 |
| cosine_precision@5 |
0.076 |
| cosine_precision@10 |
0.06 |
| cosine_recall@1 |
0.05 |
| cosine_recall@3 |
0.24 |
| cosine_recall@5 |
0.38 |
| cosine_recall@10 |
0.6 |
| cosine_ndcg@10 |
0.2931 |
| cosine_mrr@10 |
0.1985 |
| cosine_map@100 |
0.2113 |
Information Retrieval
| Metric |
Value |
| cosine_accuracy@1 |
0.05 |
| cosine_accuracy@3 |
0.28 |
| cosine_accuracy@5 |
0.36 |
| cosine_accuracy@10 |
0.55 |
| cosine_precision@1 |
0.05 |
| cosine_precision@3 |
0.0933 |
| cosine_precision@5 |
0.072 |
| cosine_precision@10 |
0.055 |
| cosine_recall@1 |
0.05 |
| cosine_recall@3 |
0.28 |
| cosine_recall@5 |
0.36 |
| cosine_recall@10 |
0.55 |
| cosine_ndcg@10 |
0.2783 |
| cosine_mrr@10 |
0.1938 |
| cosine_map@100 |
0.2078 |
Information Retrieval
| Metric |
Value |
| cosine_accuracy@1 |
0.04 |
| cosine_accuracy@3 |
0.21 |
| cosine_accuracy@5 |
0.3 |
| cosine_accuracy@10 |
0.53 |
| cosine_precision@1 |
0.04 |
| cosine_precision@3 |
0.07 |
| cosine_precision@5 |
0.06 |
| cosine_precision@10 |
0.053 |
| cosine_recall@1 |
0.04 |
| cosine_recall@3 |
0.21 |
| cosine_recall@5 |
0.3 |
| cosine_recall@10 |
0.53 |
| cosine_ndcg@10 |
0.2516 |
| cosine_mrr@10 |
0.1667 |
| cosine_map@100 |
0.1772 |
Training Details
Training Dataset
Unnamed Dataset
- Size: 900 training samples
- Columns:
positive and anchor
- Approximate statistics based on the first 1000 samples:
|
positive |
anchor |
| type |
string |
string |
| details |
- min: 159 tokens
- mean: 446.78 tokens
- max: 512 tokens
|
- min: 9 tokens
- mean: 22.04 tokens
- max: 82 tokens
|
- Samples:
| positive |
anchor |
issues related to the organization's privacy officers, exemption from consent requirements, biometric information registration, and breach reports. The next two stages will come into effect in September 2023 and September 2024, respectively.
### Hong Kong
#### Hong Kong Personal Data (Privacy) Ordinance (PDPO)
Effective Date : Since 1995 Region : APAC (Asia-Pacific)
The PDPO is the primary legislation in Hong Kong which was enacted to protect the privacy of individuals’ personal data, and regulate the collection, holding, processing, disclosure, or use of personal data by the organizations. The PDPO applies to private and public sector organizations that process, use, hold, or collect personal data. It covers any organization that deals with the collection and processing of personal data irrespective of the location of processing, provided that the personal data is controlled by the data user based in Hong Kong.
Resources*
:
Hong Kong PDPO Overview
### Ireland
#### Irish Data Protection Act (DPA)
Effective Date : May 24, 2018 Region : EMEA (Europe, the Middle East and Africa)
The Irish DPA implements the GDPR into the national law by incorporating most of the provisions of the GDPR with limited additions and deletions. It contains several provisions restricting data subjects’ rights that they generally have under the GDPR, for example, where restrictions are necessary for the enforcement of civil law claims.
Resources*
:
Irish DPA Overview
Irish Cookie Guidance
### Japan
#### Japan’s Act on the Protection of Personal Information (APPI)
Effective Date (Amended APPI) : April 01, 2022 Region : APAC (Asia-Pacific)
Japan’s APPI regulates personal related information and applies to any Personal Information Controller (the “PIC''), that is a person or entity providing personal related information for use in business in Japan. The APPI also applies to the foreign PICs which handle personal information of data subjects (“principals”) in Japan for the purpose of supplying goods or services to those persons.The act ensures the individual’s rights to privacy and also the legal use of personal data for economic development.
Resources*
:
Japan APPI Overview
### New Zealand
#### New Zealand |
What are the regulations regarding breach reports in New Zealand? |
data. Finally, as previously mentioned, consumers can opt-out of the collection of their sensitive personal data.
Means to submit DSR request:
A consumer may exercise a right by submitting an authenticated request to a controller, by means prescribed by the controller, specifying the right the consumer intends to exercise. In the instance of processing personal data concerning a child, the parent or legal guardian of the child can exercise a right on the child's behalf. In the case of processing personal data concerning a consumer subject to guardianship, conservatorship, or other protective arrangements under Title 75, Chapter 5, Protection of Persons Under Disability and Their Property, the guardian or the conservator of the consumer shall exercise a right on the consumer's behalf.
Time period to fulfill DSR request
: A controller shall comply with a consumer's request to exercise a right within 45 days after the day on which a controller had received that particular request. The controller then shall take action on the consumer's request; and inform the consumer of any action taken on the consumer's request.
Extension in the time period:
An additional 45 days can be granted if it is reasonably necessary to comply with the request, keeping in mind the complexity of the request or the volume of the requests received by the controller. In such cases, the controller is to inform the consumer of the extension and provide reasons for the extension.
Charges:
Controllers are not allowed to charge a fee for responding to a request under the law apart from certain situations. If the request is a consumer's second or subsequent request within the same 12
month period, a controller may charge a reasonable fee. A controller may also charge a reasonable fee to cover the administrative costs of complying with a request or refuse to act on a request if:
the request is excessive, repetitive, technically infeasible as per the law; or
the controller considers that the primary goal for the submitted request was something other than exercising a right; or
the request disrupts or imposes an undue burden on the resources of the controller’s business.
Appeal against refusal:
The data controller may choose to not to take action on a consumer’s DSR request. It must provide the consumer the reasons for which it did not take the action within the 45 days time period of receiving the DSR request. The data controller may also choose to not honor the request |
What is the time frame for a controller to fulfill a consumer's request to exercise a right, and what can extend this period? |
or use of personal data. This is the same as the term 'data controller.'
## Data Processor
Data Processor is a person or entity who processes personal data on behalf of another person or entity (a data user) instead of for his/her purpose(s).
## Consent
Consent is not a prerequisite for collecting personal data unless the personal data is used for a new purpose or for direct marketing purposes. Where consent is required, consent means to express and voluntary consent.
## Data Subjects' Rights under the PDPO:
The PDPO prescribes the following rights for the data subjects;
DPP 6 provides data subjects with the right to request access to and correction of their personal data. A data user should give reasons when refusing a data subject’s request to access or correction of his/her personal data.
Data subjects have the right to be informed by data user(s) regarding the holding of their personal data.
There is no explicit right to erasure available under the PDPO, however, data subjects can request the data user to delete his/her personal data that is no longer necessary for the processing. Also, data users are not allowed to retain personal data longer than necessary.
Under the PDPO, there is no right to object to processing (including profiling) available, but data subjects may opt
out from direct marketing activities.
## Who needs to comply with the PDPO?
The PDPO applies to private and public sector organizations that process, use, hold, or collect personal data. It covers any organization that deals with the collection and processing of personal data irrespective of the location of processing provided that the personal data is controlled by the data user based in Hong Kong.
The PDPO provides the following exemptions for the processing of personal data in Part VIII;
specified public or judicial interests
domestic or recreational purposes, or for
employment purposes.
The PDPO does not directly regulate data processors; therefore, they do not directly come under the application scope of the PDPO. However, data users are required to, by contractual or other means, ensure that their data processors meet the applicable requirements of the PDPO.
## Organizations' obligations under the PDPO:
PDPO does not explicitly state accountability principles and other privacy management related measures; however, the PCPD recommends |
What rights do data subjects have under the PDPO regarding the right to object to processing, and what are the limitations? |
- Loss:
MatryoshkaLoss with these parameters:{
"loss": "MultipleNegativesRankingLoss",
"matryoshka_dims": [
768,
512,
256,
128,
64
],
"matryoshka_weights": [
1,
1,
1,
1,
1
],
"n_dims_per_step": -1
}
Training Hyperparameters
Non-Default Hyperparameters
eval_strategy: epochper_device_train_batch_size: 32per_device_eval_batch_size: 16gradient_accumulation_steps: 2learning_rate: 2e-05num_train_epochs: 2lr_scheduler_type: cosinewarmup_ratio: 0.1bf16: Truetf32: Trueload_best_model_at_end: Trueoptim: adamw_torch_fusedbatch_sampler: no_duplicates
All Hyperparameters
Click to expand
overwrite_output_dir: Falsedo_predict: Falseeval_strategy: epochprediction_loss_only: Trueper_device_train_batch_size: 32per_device_eval_batch_size: 16per_gpu_train_batch_size: Noneper_gpu_eval_batch_size: Nonegradient_accumulation_steps: 2eval_accumulation_steps: Nonelearning_rate: 2e-05weight_decay: 0.0adam_beta1: 0.9adam_beta2: 0.999adam_epsilon: 1e-08max_grad_norm: 1.0num_train_epochs: 2max_steps: -1lr_scheduler_type: cosinelr_scheduler_kwargs: {}warmup_ratio: 0.1warmup_steps: 0log_level: passivelog_level_replica: warninglog_on_each_node: Truelogging_nan_inf_filter: Truesave_safetensors: Truesave_on_each_node: Falsesave_only_model: Falserestore_callback_states_from_checkpoint: Falseno_cuda: Falseuse_cpu: Falseuse_mps_device: Falseseed: 42data_seed: Nonejit_mode_eval: Falseuse_ipex: Falsebf16: Truefp16: Falsefp16_opt_level: O1half_precision_backend: autobf16_full_eval: Falsefp16_full_eval: Falsetf32: Truelocal_rank: 0ddp_backend: Nonetpu_num_cores: Nonetpu_metrics_debug: Falsedebug: []dataloader_drop_last: Falsedataloader_num_workers: 0dataloader_prefetch_factor: Nonepast_index: -1disable_tqdm: Falseremove_unused_columns: Truelabel_names: Noneload_best_model_at_end: Trueignore_data_skip: Falsefsdp: []fsdp_min_num_params: 0fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}fsdp_transformer_layer_cls_to_wrap: Noneaccelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}deepspeed: Nonelabel_smoothing_factor: 0.0optim: adamw_torch_fusedoptim_args: Noneadafactor: Falsegroup_by_length: Falselength_column_name: lengthddp_find_unused_parameters: Noneddp_bucket_cap_mb: Noneddp_broadcast_buffers: Falsedataloader_pin_memory: Truedataloader_persistent_workers: Falseskip_memory_metrics: Trueuse_legacy_prediction_loop: Falsepush_to_hub: Falseresume_from_checkpoint: Nonehub_model_id: Nonehub_strategy: every_savehub_private_repo: Falsehub_always_push: Falsegradient_checkpointing: Falsegradient_checkpointing_kwargs: Noneinclude_inputs_for_metrics: Falseeval_do_concat_batches: Truefp16_backend: autopush_to_hub_model_id: Nonepush_to_hub_organization: Nonemp_parameters: auto_find_batch_size: Falsefull_determinism: Falsetorchdynamo: Noneray_scope: lastddp_timeout: 1800torch_compile: Falsetorch_compile_backend: Nonetorch_compile_mode: Nonedispatch_batches: Nonesplit_batches: Noneinclude_tokens_per_second: Falseinclude_num_input_tokens_seen: Falseneftune_noise_alpha: Noneoptim_target_modules: Nonebatch_eval_metrics: Falsebatch_sampler: no_duplicatesmulti_dataset_batch_sampler: proportional
Training Logs
| Epoch |
Step |
Training Loss |
dim_128_cosine_map@100 |
dim_256_cosine_map@100 |
dim_512_cosine_map@100 |
dim_64_cosine_map@100 |
dim_768_cosine_map@100 |
| 0.6897 |
10 |
8.029 |
- |
- |
- |
- |
- |
| 0.9655 |
14 |
- |
0.2004 |
0.2241 |
0.2170 |
0.1726 |
0.2279 |
| 1.3793 |
20 |
5.6389 |
- |
- |
- |
- |
- |
| 1.931 |
28 |
- |
0.2078 |
0.2113 |
0.2215 |
0.1772 |
0.2441 |
- The bold row denotes the saved checkpoint.
Framework Versions
- Python: 3.10.14
- Sentence Transformers: 3.0.1
- Transformers: 4.41.2
- PyTorch: 2.1.2+cu121
- Accelerate: 0.31.0
- Datasets: 2.19.1
- Tokenizers: 0.19.1
Citation
BibTeX
Sentence Transformers
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
MatryoshkaLoss
@misc{kusupati2024matryoshka,
title={Matryoshka Representation Learning},
author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
year={2024},
eprint={2205.13147},
archivePrefix={arXiv},
primaryClass={cs.LG}
}
MultipleNegativesRankingLoss
@misc{henderson2017efficient,
title={Efficient Natural Language Response Suggestion for Smart Reply},
author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
year={2017},
eprint={1705.00652},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
