Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs: how to enable custom tools guide #1652

Merged
merged 24 commits into from
Dec 16, 2024

Conversation

itisallgood
Copy link
Contributor

@itisallgood itisallgood commented Dec 5, 2024

Copy link
Collaborator

@aantn aantn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some comments on docs

docs/configuration/ai-analysis.rst Outdated Show resolved Hide resolved
docs/configuration/ai-analysis.rst Outdated Show resolved Hide resolved
docs/configuration/ai-analysis.rst Outdated Show resolved Hide resolved
docs/configuration/ai-analysis.rst Outdated Show resolved Hide resolved
docs/configuration/ai-analysis.rst Outdated Show resolved Hide resolved
docs/configuration/ai-analysis.rst Outdated Show resolved Hide resolved
docs/configuration/ai-analysis.rst Outdated Show resolved Hide resolved
description: "Perform a GET request to example.com"
command: "curl -X GET https://example.com"
- name: "curl_with_params"
description: "Perform a GET request to example.com with query parameters"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would give a description that helps the LLM understand when to use this (use this to xyz...)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated descriptions

docs/configuration/ai-analysis.rst Outdated Show resolved Hide resolved
command: "kubectl drain {{ node_name }} --ignore-daemonsets --force --delete-emptydir-data"


Adding Custom Tools to Holmes
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the same as the two examples above?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basically yes, but previous toolsets were just examples, but here we give a short guide on how to add a toolset

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you give full insttructions on the previous 2 examples as well? (And the exact yaml structure necessary - e.g. holmes.toolsets and not just toolsets)

So we will have

Example 1 ...
Example 2 ...
Example 3 ...

And each will show the full instructions

@itisallgood itisallgood requested a review from aantn December 10, 2024 16:31
Copy link
Collaborator

@aantn aantn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good. Left a few more comments. This should be the final review, I hope.

command: "kubectl drain {{ node_name }} --ignore-daemonsets --force --delete-emptydir-data"


Adding Custom Tools to Holmes
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you give full insttructions on the previous 2 examples as well? (And the exact yaml structure necessary - e.g. holmes.toolsets and not just toolsets)

So we will have

Example 1 ...
Example 2 ...
Example 3 ...

And each will show the full instructions

docs/configuration/ai-analysis.rst Outdated Show resolved Hide resolved
docs/configuration/ai-analysis.rst Outdated Show resolved Hide resolved
- No
* - ``additional_instructions``
- string
- Additional shell commands or processing instructions applied to the output of this tool.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you give an example of how this would be used?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the description

Toolset Fields
--------------

.. list-table::
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

- "GITHUB_TOKEN"
- command: "curl --version"
tools:
- name: "list_user_repos"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

explain in a comment that parameters here are inferred

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added comments

description: "Shows the most recent commits for a repository"
command: "cd {{ repo_dir }} && git log -{{number_of_commits}} --oneline"

- name: "get_repo_details"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

explain in a comment that here you chose to be explicit about parameters - maybe show an example where this is actually useful - e.g. if you add a parameter that you want to be optional

tools:
- name: "list_user_repos"
description: "Lists all repositories for a GitHub user"
command: "curl -H 'Authorization: token {{ github_token }}' https://api.github.com/users/{{ username }}/repos"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you need to override the user_description so that the github_token is not shown there?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was changed as we removed variables section

@itisallgood itisallgood requested a review from aantn December 16, 2024 07:47
aantn
aantn previously approved these changes Dec 16, 2024
Copy link
Collaborator

@aantn aantn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left some final comments. Please fix then merge. No need for a further review.
Thanks!

- string
- Instructions on how to install prerequisites required by the toolset.
- No
* - ``variables``
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is no longer relevant, right?

- **Yes**
* - ``command``
- string
- A shell command template that the tool will execute. Can include variables and parameters using Jinja2 syntax (``{{ variable_name }}``).
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can include environment variables using ${ENV_VAR} or parameters that the LLM will fill in, using Jinja2 syntax ({{ param_name }})

@itisallgood itisallgood enabled auto-merge (squash) December 16, 2024 09:45
@itisallgood itisallgood merged commit 7e5d9a0 into master Dec 16, 2024
7 checks passed
@itisallgood itisallgood deleted the holmes-docs-enable-custom-tools branch December 16, 2024 10:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants