What are the correct TextGen WebUI settings?

[colaborat0r]

I realized that I don't actually know what settings to use on the "model" page. I have a 12GB 3060 GPU but I was running on CPU all along. I only realized that when I tried to run a larger model and it slowed to a crawl. Also I wasn't using the GBNF grammar.
Can someone provide the correct settings and recommendations for configuring TextGen WebUI to run Home-llm?

[acon96]

The only things that I set are:

Model Tab:

1. n_gpu_layers: slider all the way to max if you have a GPU

Parameters Tab:

1. temperature: 0.1
2. top_k: 40
3. Select the GBNF grammar file by either pasting it into the box or picking the file

[markmghali]

for number 3 what GBNF Grammer file?

[colaborat0r]

This
https://github.com/acon96/home-llm/blob/develop/custom_components/llama_conversation/output.gbnf

[colaborat0r]

Thanks.
Some context: After HA updates I lost the ability to control devices, and HACS components disappeared from the list but were still installed.
n_gpu_layers to max was causing errors running the model, but setting to <33 for my GPU worked.
I will test properly after a complete HA and TextGen reinstall.

[Tastaturpilot]

Hi guys, what does that error mean? How to get rid of it? Thank you!