Lora
Hi,
I was just curious if there's a reason why you chose to extract the lora with mergekit against the the base model instead of just using the lora crestfall provides in his repo?
I was curious about this too but I think I figured it out.
Sunfall is trained from the instruct model. By running the extraction using Base, fewer "instruct-isms" should be present, leaving more of the model's characteristics.
...I think. Fizz'll probably correct me.
@WesPro
@inflatebot
The reason is that Sunfall was trained on the instruct model, and Rosier was trained on the base model. I tried just directly merging the Sunfall lora, but it didn't work very well because it wasn't being applied on a base model.
So, what I did do solve this, was extract a lora from the merged Sunfall model, so that both a bit of the instruct following and a bit of the RP would be in that same lora (effectively)