Hello all! I made a pair of pregnancy fiction focused LLMs, specialized in RP in particular.
Eileithyia-7B is a 7 billion parameter model based on Mistral 7B. Eileithyia-13B is a 13 billion parameter model based on Llama 2 (via TiefighterLR).
Don't let the jargon spook you, these days running your own LLM at home is easier than ever, and I set things up so you don't even need a graphics card anymore! The skill ceiling is still high, but the floor is a lot lower.
Just install Oobabooga WebUI by downloading this ZIP. Run 'start_windows.bat', follow the simple prompts about your hardware, and you're installed. It will start a program you can access in your browser, at 127.0.0.1:7860 (or the link given in the CMD box)
Now, you need to download one of the models. For a beginner to such things, I recommend the smaller 7B model, at the 5 bit ("q5_k_m") size. This basically means it has been shrunk down to about a quarter of the original size, with minimal quality loss. It will use about 10GB of system RAM (or VRAM).
The BLUE box on the guide shows the model tab. When it loads, the UI will be in chat mode, so click into the Model tab to see the screen in the example. The GREEN box on the guide shows how to download the 5 bit 7B model. Be sure to fill out both boxes, because without the bottom box you'll download all 3 sizes!
Once the download is complete, use the "Model" dropdown (RED box) to select the model. At this point, the screen should look just like the demo, and you can click "Load" to the right of the dropdown and load the model (WHITE box). Then, the AI will speak to you back on the first page, though you may want to make some adjustments over time. If you have a graphics card, you can increase "n-gpu-layers" (YELLOW box) to move part or all of the model (remember, ~10GB of RAM/VRAM total) to your GPU's VRAM, speeding the AI up by a very VERY large amount.
Eileithyia-7B is a 7 billion parameter model based on Mistral 7B. Eileithyia-13B is a 13 billion parameter model based on Llama 2 (via TiefighterLR).
Don't let the jargon spook you, these days running your own LLM at home is easier than ever, and I set things up so you don't even need a graphics card anymore! The skill ceiling is still high, but the floor is a lot lower.
Just install Oobabooga WebUI by downloading this ZIP. Run 'start_windows.bat', follow the simple prompts about your hardware, and you're installed. It will start a program you can access in your browser, at 127.0.0.1:7860 (or the link given in the CMD box)
Now, you need to download one of the models. For a beginner to such things, I recommend the smaller 7B model, at the 5 bit ("q5_k_m") size. This basically means it has been shrunk down to about a quarter of the original size, with minimal quality loss. It will use about 10GB of system RAM (or VRAM).
The BLUE box on the guide shows the model tab. When it loads, the UI will be in chat mode, so click into the Model tab to see the screen in the example. The GREEN box on the guide shows how to download the 5 bit 7B model. Be sure to fill out both boxes, because without the bottom box you'll download all 3 sizes!
Once the download is complete, use the "Model" dropdown (RED box) to select the model. At this point, the screen should look just like the demo, and you can click "Load" to the right of the dropdown and load the model (WHITE box). Then, the AI will speak to you back on the first page, though you may want to make some adjustments over time. If you have a graphics card, you can increase "n-gpu-layers" (YELLOW box) to move part or all of the model (remember, ~10GB of RAM/VRAM total) to your GPU's VRAM, speeding the AI up by a very VERY large amount.
