feat: Add Query Trace Deleter #11026
Conversation
facebook-github-bot
added
the
CLA Signed
✅ Deploy Preview for meta-velox canceled.
|
xiaodouchen
force-pushed
the
extend_trace_delete
branch
from
9bce80e to
ee27d06
Compare
|
@duanmeng @xiaoxmeng @mbasmanova I have attempted to add a deletion module. Could you please review it? |
|
This pull request has been automatically marked as stale because it has not had recent activity. If you'd still like this PR merged, please comment on the PR, make sure you've addressed reviewer comments, and rebase on the latest main. Thank you for your contributions! |
|
@xiaodouchen Hi Xiaodou, thanks for adding this feature, it is very useful. Do you still have the capacity and interest to pursue it? If so, please reopen this PR and rebase it so we can begin the review and merge process. cc @xiaoxmeng |
@duanmeng Yes, I will rebase it. |
xiaodouchen
force-pushed
the
extend_trace_delete
branch
from
ee27d06 to
57a5e47
Compare
| "Query trace delete enabled but the trace dir is not set"); | ||
|
|
||
| const auto deleter = trace::OperatorTraceDeleter::instance(); | ||
| deleter->asyncDeleteDir(queryConfig.queryTraceDir()); |
There was a problem hiding this comment.
Do you mean to delete the trace data by submitting a query?
There was a problem hiding this comment.
Yes, delete the trace data on the node where the query is executed.
There was a problem hiding this comment.
This is kind of overkill. Normally, we store the tracing data in remote storage systems. I am adding a trace file operation tool; maybe you can add the delete function to it after it is landed. What do you think? @xiaodouchen @xiaoxmeng
xiaodouchen
force-pushed
the
extend_trace_delete
branch
from
57a5e47 to
4d01f5a
Compare
Add a query trace deleter to delete all generated trace data in `query_trace_dir` and avoid manually deleting them on each node of clusters. Note that for the same `query_trace_dir`, the generation and deletion of trace files should not occur concurrently to avoid potential conflicts or issues.
xiaodouchen
force-pushed
the
extend_trace_delete
branch
from
4d01f5a to
d219aae
Compare
Add a query trace deleter to delete all generated trace data in
query_trace_dirand avoid manually deleting them on each node ofclusters. Note that for the same
query_trace_dir, the generation anddeletion of trace files should not occur concurrently to avoid potential
conflicts or issues.
Part of #9668