Update README.md

README.md

@@ -44,7 +44,7 @@ We explore supervised multitask pre-training by proposing ***Instruction Pre-Tra
 ## Domain-Adaptive Continued Pre-Training
 Following [AdaptLLM](https://huggingface.co/AdaptLLM/finance-chat), we augment the domain-specific raw corpora with instruction-response pairs generated by our [context-based instruction synthesizer](https://huggingface.co/instruction-pretrain/instruction-synthesizer).
 
-### 1. To chat with the finance-Llama3-8B model:
+### 1. chat with the finance-Llama3-8B model:
 ```python
 from transformers import AutoModelForCausalLM, AutoTokenizer
 
@@ -70,30 +70,43 @@ pred = tokenizer.decode(outputs[answer_start:], skip_special_tokens=True)
 print(pred)
 ```
 
-### 2. To evaluate our models on the domain-specific tasks
-1. Set up dependencies
-```bash
-git clone https://github.com/microsoft/LMOps
-cd LMOps/adaptllm
-pip install -r requirements.txt
-```
-
-2. Evaluate
-```bash
-DOMAIN='finance'
-
-# if the model can fit on a single GPU: set MODEL_PARALLEL=False
-# elif the model is too large to fit on a single GPU: set MODEL_PARALLEL=True
-MODEL_PARALLEL=False
+### 2. evaluate any Huggingface LMs on domain-dpecific tasks (💡New!)
+You can use the following scripts to reproduce our results and evaluate any other Huggingface models on domain-specific tasks. Note that these scripts are not applicable to models that require specific prompt templates (e.g., Llama2-chat, Llama3-Instruct).
+
+1). Set Up Dependencies
+   ```bash
+   git clone https://github.com/microsoft/LMOps
+   cd LMOps/adaptllm
+   pip install -r requirements.txt
+   ```
+
+2). Evaluate the Model
+   ```bash
+   # Select the domain from ['biomedicine', 'finance', 'law']
+   DOMAIN='finance'
+  
+   # Specify any Huggingface LM name (Not applicable to models requiring specific prompt templates)
+   MODEL='instruction-pretrain/finance-Llama3-8B'
+  
+   # Model parallelization:
+   # - Set MODEL_PARALLEL=False if the model fits on a single GPU. 
+   #   We observe that LMs smaller than 10B always meet this requirement.
+   # - Set MODEL_PARALLEL=True if the model is too large and encounters OOM on a single GPU.
+   MODEL_PARALLEL=False
+  
+   # Choose the number of GPUs from [1, 2, 4, 8]
+   N_GPU=1
+  
+   # Whether to add a BOS token at the beginning of the prompt input:
+   # - Set to False for AdaptLLM.
+   # - Set to True for instruction-pretrain models.
+   # If unsure, we recommend setting it to False, as this is suitable for most LMs.
+   add_bos_token=True
+
+   # Run the evaluation script
+   bash scripts/inference.sh ${DOMAIN} ${MODEL} ${add_bos_token} ${MODEL_PARALLEL} ${N_GPU}
+   ```
 
-# number of GPUs, chosen from [1,2,4,8]
-N_GPU=1
-
-# Set as True
-add_bos_token=True
-
-bash scripts/inference.sh ${DOMAIN} 'instruction-pretrain/finance-Llama3-8B' ${add_bos_token} ${MODEL_PARALLEL} ${N_GPU}
-```
 
 ## Citation
 If you find our work helpful, please cite us: