LM Studio

[eponce92]

I am trying to setup home-llm with the server running on LM Studio, I have selected "generic OpenAI API" , set the API Hostname as the IP of my PC running LM Studio server , not local host or anything like that. API port as 1234, Model Name as "acon96/Home-3B-v2-GGUF" and API Key as "not-needed" (also tried with leaving it empty). Keep getting "Unexpected error during intent recognition" on the Assist chat interface as the response for any message. Any ideas what I'm doing wrong here ?

[markmghali]

that is more on the home assistant side, I think. Trying to add as many aliases as you can per device and making sure your entities are exposed to assist.

[eponce92]

I got it to communicate with the server on LM Studio. I have it exposed to only one light by using a helper called : living , light.living
Get this response when I just tell it to Turn off the living room:
I'll turn it of for you now. Failed to parse call from '{"service": "device_control", "target_ device": {"name":"light.living"} }'!

[markmghali]

what prompt are you using? Also I think acon96 was saying that the v2 model got messed up abit and is redoing it for v3

[eponce92]

so in LM studio you cannot change the prompt so I think its done on the background by the LlaMA conversation integration. I can see on the server log the following as the first message :

[2024-02-01 13:46:54.949] [INFO] Received POST request to /v1/completions with body: {
"model": "acon96/Home-3B-v2-GGUF",
"max_tokens": 128,
"temperature": 0.1,
"top_p": 1,
"prompt": "<|im_start|>system\nYou are 'Al', a helpful AI Assistant that controls the devices in a house. Complete the following task ask instructed with the information provided only.\nServices: light.turn_on(brightness,rgb_color), light.turn_off(), light.toggle(brightness,rgb_color)\nDevices:\nlight.the_living_room 'the living room' = off<|im_end|>\n<|im_start|>user\nturn on the lving room<|im_end|>\n<|im_start|>assistant"
}
[2024-02-01 13:46:54.950] [INFO] Provided inference configuration: {
"n_threads": 4,
"n_predict": 128,
"top_k": 40,
"min_p": 0.05,
"top_p": 1,
"temp": 0.1,
"repeat_penalty": 1.5,
"input_prefix": "Instruct: ",
"input_suffix": "\nOutput:",
"antiprompt": [
"Instruct:",
"Output:"
],
"pre_prompt": "",
"pre_prompt_suffix": "",
"pre_prompt_prefix": "",
"seed": -1,
"tfs_z": 1,
"typical_p": 1,
"repeat_last_n": 64,
"frequency_penalty": 0,
"presence_penalty": 0,
"n_keep": 0,
"logit_bias": {},
"mirostat": 0,
"mirostat_tau": 5,
"mirostat_eta": 0.1,
"memory_f16": true,
"multiline_input": false,
"penalize_nl": true
}

[markmghali]

I am having the same issue on textgen webui with the same error spitting out immediately. I am trying to figure it out as well

[markmghali]

yeah you need to update the prompt in the llama conversation integration with the device and services you are exposing.

[markmghali]

I fixed my issue by using the develop branch
#52