Update README.md

README.md

@@ -33,6 +33,25 @@ The maximum GPU usage during training is **24GB**, and the model has preliminary
 
 This model is fine-tuned using 1,000 examples of the Alpaca-GPT4 and Glaive-function-calling-v2 datasets, respectively.
 
+## Usage
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer
+from peft import PeftModel
+
+tokenizer = AutoTokenizer.from_pretrained("hiyouga/Llama-2-70b-AQLM-2Bit-QLoRA-function-calling")
+model = AutoModelForCausalLM.from_pretrained("BlackSamorez/Llama-2-70b-AQLM-2Bit-1x16-hf", torch_dtype="auto", device_map="auto")
+model = PeftModel.from_pretrained(model, "hiyouga/Llama-2-70b-AQLM-2Bit-QLoRA-function-calling")
+streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)
+
+messages = [
+    {"role": "user", "content": "Who are you?"}
+]
+inputs = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt")
+inputs = inputs.to("cuda")
+generate_ids = model.generate(inputs, streamer=streamer)
+```
+
 ## Training procedure
 
 ### Training hyperparameters