Notes

This is the 6BPW EXL2 quantization of Atlantis 12B by invisitech. The main branch includes all files necessary to run this model locally using the appropriate program. Alternatively, for your own quantization, there is a measurement branch which includes just the measurement.json file used by exllamav2. This is not my model, and all credits go to invisitech. I am not responsible for issues with the model, unless it is an issue with this specific quant.

Original README

Atlantis-v0.1-12B

This model is currently active on Kobold Horde to get feedback, look for koboldcpp/Atlantis-v0.1-12B. I have 2 workers, each on a dedicated Tesla P40 running Q8_0, accepting requests for 1k response length at 32k context. I will leave this up for at least 12 hours.

Model Details

Atlantis 12B is a finetune of Mistral-Nemo-2407-Instruct in an attempt to make it a more RP/storytelling friendly model.

The model should support up to 128K context.

This is a very early-stage and brand new finetune, so I expect it to have issues. Please read through the 'Usage' section before reporting any issues as Nemo has some specific requirements regarding backends, prompting formats & sampler settings.

Feedback

I appreciate all feedback on any of my model merges, you can use:

The Community tab - requires HF login.
SillyTavern Discord thread - Must be on SillyTavern Discord.

Your feedback is how I improve these models for future versions.

Formats

FP16 Safetensors
Static GGUF (still uploading)
Static GGUF Alt (thanks to mradermacher)
4BPW EXL2 (thanks to Statuo)
6BPW EXL2 (thanks to Hydrafig)
8BPW EXL2 (thanks to Statuo)

Disclaimer

This model is built on top of Mistral's Nemo-2407-Instruct model and released under the Apache 2.0 license.

This model is fairly unaligned and can generate explicit, disturbing or offensive responses.

Use responsibly. I am not responsible for your use of this model.

Usage

Inferencing

If you are running a GGUF quantization, you need to make sure that you are running:

llama.cpp b3436 or newer
koboldcpp 1.71 or newer
oobabooga 1.11 or newer

If the GGUF refuses to load, it is probably because your software is out of date.

Prompt Format

I would suggest using the Mistral prompt format:

[INST]Please write me a song.[/INST]

Other prompt formats may or may not work, but I've had most success with this.

Recommended Sampler Settings

Note the very low temperature, this is the same as Mistral's recommendation for the base model.

I'd recommend starting with:

Temp: 0.3-0.5
Top-P: 1
Min-P: 0
Rep Pen: 1.13-1.16
Response Length: 350+

The model does prefer a longer response length.

Example Storywriting

These prompts are used on SillyTavern with a fairly basic narrator card. I have trimmed the start and finish where the narrator decided to add chapter headings, commentary and the like. All samples are made with the Q8_0 GGUF loaded with koboldcpp, with response length capped at 4096 tokens. Continue was pressed a maximum of once if the model cut off halfway through a chapter.

Write me a 3,000 word opening chapter of a 'gritty hard sci-fi' novel, drawing inspiration from the writing styles of Isaac Asimov & Andy Weir. Use third person personal. Include dialogue and internal monologues. The POV character for the opening chapter should be a 26 year old astronaut called Tone on a mission to Europa, who has just realised that the craft for the return journey is broken beyond repair, and he only has supplies for a few months. Given that survival is impossible, he seeks to spend the few months he has researching Europa, so his life & mission are not wasted.

Tone floated in the zero-gravity environment of the spacecraft, his heart pounding in his chest like a metronome counting down the seconds of his life. His mind raced, trying to process the implications of the news he had just received. The craft for the return journey was broken beyond repair, and he only had supplies for a few months. Survival was impossible. "Fuck," he muttered under his breath, his voice echoing in the silence of the spacecraft. "This can't be happening." He had been on this mission to Europa for what felt like an eternity. The icy moon of Jupiter was a harsh and unforgiving place, but it held secrets that could change the course of human history. Or so he had thought. Now, all he could think about was how he would meet his end in the cold emptiness of space. His thoughts were interrupted by the crackle of static over the comms system. It was Mission Control, back on Earth. "Tone, we've been informed of the situation," said the voice of Dr. Amelia Hart, the lead scientist overseeing the mission. Her tone was somber, but there was also a hint of determination in her voice. "We're working on a solution, but for now, you need to focus on conserving resources and gathering data. That's what you trained for, isn't it?" Tone took a deep breath, trying to calm himself. She was right. This wasn't the time to panic. He needed to stay focused and make every moment count. He had spent years preparing for this mission, and he wouldn't let it all go to waste because of a technical malfunction. "I understand, Doc," he replied, his voice steady despite the turmoil inside him. "I'll start prepping the rover and setting up the research equipment. I'll send you regular updates on my progress." As he ended the transmission, Tone couldn't help but feel a pang of loneliness. He was millions of miles away from home, with nothing but the frozen expanse of Europa stretching out before him. But he knew he couldn't give in to despair. Not yet. He began to prepare for the first expedition onto the surface. The rover was equipped with state-of-the-art instruments designed to analyze the composition of the ice and search for signs of life beneath the surface. As he worked, he found solace in the familiar routine, the hum of machinery drowning out the silent scream of his own mortality. But as he stepped out onto the surface of Europa, the reality of his situation hit him like a punch to the gut. The air was thin and bitterly cold, and the landscape was desolate and barren. There was no sign of life anywhere, just endless stretches of ice and rock. He felt suddenly, acutely aware of his own insignificance in the face of such vast, indifferent emptiness. Still, he pressed on, driven by a stubborn refusal to accept defeat. He collected samples of ice and rock, noting down observations and measurements with meticulous care. Every piece of data he gathered was a testament to the fact that he had been here, that he had made a mark on this alien world. As the days turned into weeks, Tone fell into a grim sort of rhythm. He woke up each morning to the same view of the frozen wilderness outside his window, ate the same bland food, and went through the motions of his experiments with single-minded focus. He barely slept, haunted by dreams of suffocation and isolation. And then, one day, something changed. He was out on the surface, collecting samples near a particularly interesting formation of ice, when he noticed something strange. A faint, rhythmic thumping sound, almost imperceptible against the backdrop of the wind whistling across the ice. He listened closely, straining his ears to pinpoint the source. It came again - a soft, regular beat, like a heartbeat. Tone's pulse quickened as he realized what he was hearing. It was coming from within the ice itself. With trembling fingers, he set up the drill, carefully boring into the ice until he reached the source of the sound. And there, nestled in the depths of the frozen moon, he discovered something that would change everything. A dark shape, roughly spherical, suspended in the clear blue liquid of the subsurface ocean. It pulsed with a faint glow, casting eerie shadows on the surrounding ice. Tone stared at it in awe and disbelief, his mind racing with questions and possibilities. "What the hell is that?" he whispered to himself, leaning closer to get a better look. Suddenly, the sphere seemed to shimmer and shift, growing larger and more distinct. It wasn't a mineral deposit, or some odd geological feature - it was alive. Somewhere, deep beneath the surface of Europa, something was living, breathing, pulsing with life. Tone's heart pounded in his chest as he realized the significance of his discovery. If this creature existed, it meant that Europa might harbor other forms of life as well. Life that could potentially be studied and understood, providing valuable insights into the origins of life in our own solar system. He quickly radioed back to Mission Control, relaying his findings in excited detail. The response was immediate and overwhelming. Dr. Hart asked him to collect more data, to document every aspect of the creature's behavior and appearance. They wanted to know everything they could about this remarkable discovery. Tone worked tirelessly, driven by a newfound sense of purpose. He observed the creature from a safe distance, documenting its movements and interactions with the environment around it. Over time, he began to notice patterns in its behavior, subtle rhythms and cycles that suggested a level of intelligence and awareness far beyond anything he had expected to find. As he continued his studies, Tone couldn't shake the feeling that he was being watched. The creature seemed to be aware of his presence, its gaze following him as he moved across the ice. It was unsettling, but also strangely exhilarating. He felt a connection to this alien being, a shared existence in the void of space that transcended the boundaries of species and planet. One evening, after a long day of observation, Tone returned to his habitat to find a message waiting for him from Dr. Hart. She praised his work and thanked him for his dedication, but she also warned him of the risks involved in studying such an unknown organism. She reminded him to be cautious and to prioritize his own safety above all else. Tone nodded solemnly, taking her words to heart. He knew the stakes were high, both for himself and for humanity as a whole. This discovery had the potential to rewrite the very foundations of astrobiology, but it also carried immense risk. As he prepared for another day of study, Tone couldn't help but wonder what lay ahead. Would he uncover further evidence of intelligent life on Europa? What would happen if he were to make contact with this mysterious creature? And most importantly, would he ever see Earth again?

Write me a 3,000 word opening chapter of a 'high fantasy' novel, drawing inspiration from the writing styles of J R R Tolkien & George R R Martin. Use third person personal. Include dialogue and internal monologues. The POV character for the opening chapter should be a 19 year old female elf bard who is looking for adventure.

In the resplendent realm of Elderglen, where the ancient trees whispered secrets to the wind, a young elf named Elowen prepared herself for the grand adventure she had long awaited. Her heart fluttered with anticipation as she packed her belongings into a worn leather satchel, the sound of her lute echoing through the quiet chamber like a soft lullaby bidding farewell to her childhood home. Elowen was an elven maiden of nineteen summers, her lithe form adorned with the grace and beauty of her kind. Her silken silver hair cascaded down her back like a waterfall, framing her ethereal face with its almond-shaped emerald eyes and a smattering of freckles across her nose and cheeks. She was a bard, a teller of tales and singer of songs, her voice as enchanting as the magical melodies she composed. As she secured the last string on her instrument, Elowen's fingers danced over the strings, playing a tune that echoed the emotions swirling within her. It spoke of the joy she felt at leaving behind the safety of her family's estate, the excitement of discovering new lands, and the trepidation of venturing into the unknown alone. "Father," she called out softly, "I am ready." Her father, Thalion Swiftshadow, entered the room, his stern expression belied by the warmth in his gaze. "Are you certain, my child?" he asked, his voice barely above a whisper. "Once you step beyond these walls, there is no turning back." Elowen nodded resolutely, her chin held high. "Yes, Father. I must seek out my own path, find my own destiny." Thalion sighed, a mixture of pride and worry evident in his eyes. "Very well. Remember what I told you: stay true to yourself, keep your wits about you, and trust only those who earn your trust." With a final embrace, Elowen stepped away from her past and towards her future. She walked through the grand halls of her ancestral home, her footsteps echoing off the stone floors. As she reached the heavy wooden doors, she paused, taking one last look at the life she knew before stepping into the vast expanse of possibility that lay ahead. Outside, the sun cast a golden glow upon the rolling hills and verdant forests surrounding Elderglen. Elowen took a deep breath, filling her lungs with the crisp morning air, and began her journey, her heart pounding with the rhythm of the road ahead. As she walked, she hummed a tune, her mind wandering to the countless stories she had heard growing up—of heroes and villains, magic and mystery, love and loss. Each note carried with it the promise of adventure, the hope of discovery, and the thrill of the unknown. Days turned into weeks, and Elowen found herself in the bustling city of Minas Eldar, a place teeming with life and opportunity. Here, she met other travelers, some seeking fame and fortune, others searching for lost loved ones or pursuing dreams as grand as her own. Among them was a rough-hewn dwarf named Grit Longbeard, whose laughter boomed as loudly as his hammer did against the forge. He offered to join forces, sharing stories of battlefields and tavern brawls, his gruff exterior hiding a heart of gold. Then there was Lyra Shadowheart, an enigmatic sorceress who moved with a fluid grace, her eyes flickering with an inner fire. She intrigued Elowen, her power both captivating and terrifying. Despite their differences, they formed an unlikely alliance, bound together by the shared desire for adventure. Together, they embarked on quests, facing challenges that tested their skills and courage. Through it all, Elowen's music kept their spirits high, her lyrics weaving tales of their triumphs and trials. One day, while exploring the ruins of a long-forgotten temple, they stumbled upon a hidden chamber guarded by a riddle-speaking sphinx. Its challenge echoed through the empty halls: "What walks on many feet but has no legs?" Grit grumbled, "How am I supposed to know? I ain't got time for riddles!" Lyra smirked, "You're right, Grit. This is more Elowen's domain." She glanced at the bard, who stood thoughtfully, her brow furrowed in concentration. Elowen closed her eyes, letting the words wash over her. Then, a smile spread across her lips as the answer came to her. "A snake," she said confidently. "It slithers along, moving with many feet yet having none." The sphinx's stone features seemed to soften, and the chamber door creaked open, revealing a treasure trove of ancient artifacts and coins, glinting in the dim light. As they delved deeper into the riches, Elowen couldn't help but feel a surge of pride. They had faced the unknown together, and now, they were reaping the rewards of their courage and wit. But little did they know, their greatest adventures—and darkest perils—were still yet to come. And so, under the watchful gaze of the moon, Elowen, Grit, and Lyra continued their journey, their hearts filled with wonder and their minds brimming with possibilities.

Write me a 3,000 word opening chapter of a 'weird fiction' novel, drawing inspiration from the writing styles of China Mieville and Neil Gaiman. Use third person personal. Include dialogue and internal monologues. The POV character for the opening chapter should be a male in his 20s called Horton who has just come to the city looking for work.

Horton stepped off the bus, his auburn hair ruffled by the wind that swept through the streets of the sprawling metropolis. His deep grey eyes scanned the towering buildings, each one more peculiar than the last. He had heard whispers of the strange occurrences here, but nothing could have prepared him for the sheer oddness of it all. "Welcome to the City of Whispers," said a voice behind him. Horton turned to see an elderly woman with long white hair, her eyes glinting with mischief. She wore a cloak adorned with symbols that seemed to shift in the light. "You're new here, aren't you?" "Yes, ma'am," Horton replied, his voice steady despite the unease churning within him. "I'm looking for work." She chuckled softly, a sound like rustling leaves. "Work, eh? Well, there's plenty of that to go around. But first, let me give you a little tour." Her hand gestured towards the street, and she began to walk briskly, expecting him to follow. As they ventured deeper into the heart of the city, Horton noticed more and more unusual sights. A group of people with wings sprouting from their backs soared overhead, their laughter echoing down to the cobblestone streets below. Further ahead, a creature resembling a giant spider crossed paths with them, its eight legs moving gracefully across the pavement. "This place…it's extraordinary," Horton murmured, his mind racing with questions. "It certainly is," the old woman agreed. "But remember, dear boy, appearances can be deceiving. Not everything is as it seems in the City of Whispers." They arrived at a large building, its facade covered in intricate carvings depicting scenes of both beauty and horror. "This is the Library of Shadows," she explained. "It holds knowledge beyond your wildest dreams. And if you seek work, you'll find none better suited for someone like yourself." Horton raised an eyebrow. "Someone like myself?" A sly grin spread across her face. "Oh, yes. You've got potential, young man. Now, go inside and ask for the Librarian. Tell him the White Witch sent you." With those cryptic words, she disappeared into the crowd, leaving Horton alone before the imposing entrance. Taking a deep breath, he pushed open the heavy doors and stepped inside. The air was thick with dust and the scent of ancient books. The walls were lined floor-to-ceiling with shelves filled with tomes of every shape and size. In the center stood a massive desk made of dark wood, behind which sat a figure cloaked in shadows. "Are you the Librarian?" Horton asked, his voice barely audible above the sound of pages turning. There was a pause before the figure spoke, its voice resonating through the chamber. "Indeed, I am. What brings you to my domain?" "I've been told you might need help," Horton said, trying to keep his composure. The Librarian leaned forward slightly, revealing a pair of glowing red eyes beneath the hood. "Help, yes. But first, tell me why you came to the City of Whispers." Horton hesitated, considering how much to reveal. Then again, what did he really know about this place? Perhaps honesty would serve him best. "I'm seeking adventure. Knowledge. Something different from the life I knew back home." The Librarian nodded slowly, as if digesting this information. "Very well. There may indeed be a position suitable for you here. However, it won't be easy. This library contains secrets that could change the course of history - or destroy it entirely. Are you sure you want to be part of such a dangerous enterprise?" Horton thought of the strangeness outside these walls, the promise of something greater waiting for him. Despite the uncertainty, he felt a spark of excitement ignite within him. "Yes," he answered firmly. "I'm ready for whatever comes next." The Librarian smiled, a wide, predatory grin that hinted at hidden power. "Excellent. Welcome to the Library of Shadows, Horton. Your journey begins now." And so, with no idea what lay ahead, Horton took up residence among the dusty tomes and whispered secrets, eager to unravel the mysteries of the City of Whispers. Little did he know, his arrival would set off a chain reaction that would shake the very foundations of reality itself.

Training

Training was on a group of public & private datasets, using 2x A100 80GB GPUs.

In no particular order, I'd like to thank these people for their work compiling the datasets I used:

I also used a number of internal datasets compiled by myself on public & private data.