as an option would be really good idea expecting release

Agreed.

If there ist no option for local api /llm server please add this

fully suppurt local LLM

What about NovelAI? I can use it for sillytavern, using it for something like this would be great.

Originally posted by pogliacci:

What about NovelAI? I can use it for sillytavern, using it for something like this would be great.

酒馆用Claude模型更好，它更适合文笔
The Claude model is better for sillytavern as it is more suitable for writing

Agreed

Originally posted by DesktopDev:

fully suppurt local LLM

Yup, we can load local models, but be great if we can choose the LLM server (instead use the embeeded one), online, offline or in another server in our LAN. (Ollama, SillyTavern, OpenWebUI...).

For example I have already ollama runing on my PC, so my VRAM it's feed with two instances when I use Chat waifu.

External servers allow us to add RAG and plugins that make the experience better (Like SillyTavern for example)

NovelAI now has a Llama 3 LLM available to Opus tier, so now's a great time to use their API.

Originally posted by Mitsuha Kitsune:

Originally posted by DesktopDev:

fully suppurt local LLM

Yup, we can load local models, but be great if we can choose the LLM server (instead use the embeeded one), online, offline or in another server in our LAN. (Ollama, SillyTavern, OpenWebUI...).

For example I have already ollama runing on my PC, so my VRAM it's feed with two instances when I use Chat waifu.

External servers allow us to add RAG and plugins that make the experience better (Like SillyTavern for example)

Even better: LM Studio.

I would suggest to try Hermes3

Silly tavern has a fantastic front end for Characters. It would be awesome to find away to get ChatWaifu to handle the "functional" tasks, to her Qwen 2.5 model for reasoning, while sending all the "role play and adventures" to Silly Tavern. Including the TTS ect.

If there are any others who are interested in this that would be awesome. We can compile our collective goon-age and get this to happen LOL!

愚蠢酒馆（Silly Tavern）在角色展示方面的前端非常棒。如果能让ChatWaifu负责“功能性”任务，例如用她的Qwen 2.5模型进行推理，同时将所有“角色扮演和冒险”部分交给愚蠢酒馆处理，那就太棒了，包括文字转语音（TTS）等功能。

如果还有其他人对这个感兴趣，那就太好了！我们可以集合大家的力量，一起让这个愿景成为现实，哈哈哈！

Forgive my Chinese. I can't speak it.

As that is the first thing I did was connect it to the local LLM I have been using with KoboldCPP I will say yes it is able to use local LLM models.

It just does not have the output progress bar in a command line nor the settings control KoboldCPP allows. Most likely due to KoboldCPP is the GUI for a backend and this is a purely front end thing. Just happy this one also uses GGUF files not GGML nor requiring a line of safetensors files to work.