santyzenith (Santiago Garcia)

upvoted an article 1 day ago

Article

Fine-tuning 20B LLMs with RLHF on a 24GB consumer GPU

Mar 9, 2023

• 30

upvoted 2 papers about 1 month ago

Compact Language Models via Pruning and Knowledge Distillation

Paper • 2407.14679 • Published Jul 19 • 35

MoLE : Mixture of Language Experts for Multi-Lingual Automatic Speech Recognition

Paper • 2302.13750 • Published Feb 27, 2023 • 2

upvoted 3 articles about 1 month ago

Article

Introduction to Graph Machine Learning

Jan 3, 2023

• 15

Article

Llama 3.1 - 405B, 70B & 8B with multilinguality and long context

Jul 23

• 193

Article

Welcome Gemma 2 - Google's new open LLM

Jun 27

• 115

upvoted a paper 2 months ago

DataComp-LM: In search of the next generation of training sets for language models

Paper • 2406.11794 • Published Jun 17 • 48

upvoted an article 2 months ago

Article

SmolLM - blazingly fast and remarkably powerful

Jul 16

• 242

upvoted an article 3 months ago

Article

From PyTorch DDP to 🤗 Accelerate to 🤗 Trainer, mastery of distributed training with ease

Oct 21, 2022

• 8

upvoted a paper 3 months ago

Tuna: Instruction Tuning using Feedback from Large Language Models

Paper • 2310.13385 • Published Oct 20, 2023 • 10

upvoted a collection 3 months ago

Knowledge distillation

Collection

88 items • Updated Feb 7 • 6

upvoted 2 articles 3 months ago

Article

Putting RL back in RLHF

Jun 12

• 58

Article

Fine-Tune Whisper with 🤗 Transformers

Nov 3, 2022

• 84

upvoted 6 papers 3 months ago

Datasets: A Community Library for Natural Language Processing

Paper • 2109.02846 • Published Sep 7, 2021 • 8

Estimating Knowledge in Large Language Models Without Generating a Single Token

Paper • 2406.12673 • Published Jun 18 • 7

A Systematic Survey of Text Summarization: From Statistical Methods to Large Language Models

Paper • 2406.11289 • Published Jun 17 • 5

upvoted an article 3 months ago

Article

Illustrating Reinforcement Learning from Human Feedback (RLHF)

Dec 9, 2022

• 75

upvoted 4 collections 3 months ago

Gemma 2 Release

Collection

15 items • Updated 10 days ago • 166

Qwen2

Collection

Qwen2 language models, including pretrained and instruction-tuned models of 5 sizes, including 0.5B, 1.5B, 7B, 57B-A14B, and 72B. • 39 items • Updated 1 day ago • 332

Probably DPO datasets

Collection

A collection of datasets that probably support DPO • 146 items • Updated Jun 26 • 12

FP8 LLMs for vLLM

Collection

Accurate FP8 quantized models by Neural Magic, ready for use with vLLM! • 37 items • Updated 23 days ago • 51

upvoted 3 articles 3 months ago

Article

Training and Finetuning Embedding Models with Sentence Transformers v3

May 28

• 146

Article

🦙⚗️ Using Llama3 and distilabel to build fine-tuning datasets

By

•

Jun 4

• 67

Article

Mergoo: Efficiently Build Your Own MoE LLM

By

•

Jun 3

• 40

upvoted a collection 4 months ago

Qwen1.5

Collection

Qwen1.5 is the improved version of Qwen, the large language model series developed by Alibaba Cloud. • 55 items • Updated 1 day ago • 205

upvoted 4 articles 5 months ago

Article

Fine-tune Llama 3 with ORPO

By

•

Apr 22

• 221

Article

Assisted Generation: a new direction toward low-latency text generation

May 11, 2023

• 19

Article

Faster fine-tuning using TRL & Unsloth

Jan 10

• 35

Article

RAG chatbot using llama3

By

•

Jul 7

• 66

Santiago Garcia

AI & ML interests

Organizations

santyzenith's activity

Fine-tuning 20B LLMs with RLHF on a 24GB consumer GPU

Compact Language Models via Pruning and Knowledge Distillation

MoLE : Mixture of Language Experts for Multi-Lingual Automatic Speech Recognition

Introduction to Graph Machine Learning

Llama 3.1 - 405B, 70B & 8B with multilinguality and long context

Welcome Gemma 2 - Google's new open LLM

DataComp-LM: In search of the next generation of training sets for language models

SmolLM - blazingly fast and remarkably powerful

From PyTorch DDP to 🤗 Accelerate to 🤗 Trainer, mastery of distributed training with ease

Tuna: Instruction Tuning using Feedback from Large Language Models

Knowledge distillation

Putting RL back in RLHF

Fine-Tune Whisper with 🤗 Transformers

Datasets: A Community Library for Natural Language Processing

Estimating Knowledge in Large Language Models Without Generating a Single Token

A Systematic Survey of Text Summarization: From Statistical Methods to Large Language Models

Adam-mini: Use Fewer Learning Rates To Gain More

Aligning Teacher with Student Preferences for Tailored Training Data Generation

Direct Preference Knowledge Distillation for Large Language Models

Illustrating Reinforcement Learning from Human Feedback (RLHF)

Gemma 2 Release

Qwen2

Probably DPO datasets

FP8 LLMs for vLLM

Training and Finetuning Embedding Models with Sentence Transformers v3

🦙⚗️ Using Llama3 and distilabel to build fine-tuning datasets

Mergoo: Efficiently Build Your Own MoE LLM

Qwen1.5

Fine-tune Llama 3 with ORPO

Assisted Generation: a new direction toward low-latency text generation

Faster fine-tuning using TRL & Unsloth

RAG chatbot using llama3