Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Adding async data cache configs and ssd cache documentation #11429

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

minhancao
Copy link
Contributor

@minhancao minhancao commented Nov 5, 2024

Add async data cache configs and ssd cache documentation.
Add instructions on how to setup btrfs filesystem for SSD usage on AWS worker machines of Linux OS.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 5, 2024
Copy link

netlify bot commented Nov 5, 2024

Deploy Preview for meta-velox canceled.

Name Link
🔨 Latest commit 99cb9d0
🔍 Latest deploy log https://app.netlify.com/sites/meta-velox/deploys/674f9acb5b319600080ec717

@minhancao minhancao changed the title doc: Adding async data cache and ssd cache documentation doc: Adding async data cache configs and ssd cache documentation Nov 5, 2024
@minhancao minhancao marked this pull request as ready for review November 6, 2024 22:43
@minhancao
Copy link
Contributor Author

@majetideepak Can you review this PR when you can? Thanks!

Copy link
Collaborator

@czentgr czentgr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this. Multiple comments.

velox/docs/configs.rst Outdated Show resolved Hide resolved
velox/docs/configs.rst Outdated Show resolved Hide resolved
velox/docs/configs.rst Outdated Show resolved Hide resolved
velox/docs/configs.rst Outdated Show resolved Hide resolved
- Min SSD savable (in-memory) cache space to start writing SSD savable cache entries into SSD cache. NOTE: we only write to SSD cache when both above conditions are satisfied.
* - async-cache-persistence-interval
- string
- 0s
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the explanation do we need to add the allowed units to be specified? Is this mentioned in other configs that use duration units?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will add units has to be in seconds here.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The unit is part of the string and we can see it is in seconds. But we would allow time based units like hour, day etc to be used as well. Not just seconds and we want to mention this in the description what they are.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added
The following time units are supported: ns, us, ms, s, m, h, d.

Reference: https://github.com/facebookincubator/velox/blob/main/velox/common/config/Config.cpp#L82

velox/docs/develop/cache.rst Outdated Show resolved Hide resolved
velox/docs/develop/cache.rst Outdated Show resolved Hide resolved
velox/docs/develop/cache.rst Outdated Show resolved Hide resolved
velox/docs/develop/cache.rst Outdated Show resolved Hide resolved
velox/docs/develop/cache.rst Outdated Show resolved Hide resolved
@minhancao minhancao changed the title doc: Adding async data cache configs and ssd cache documentation docs: Adding async data cache configs and ssd cache documentation Nov 19, 2024
@ethanyzhang
Copy link

@czentgr to re-review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants