Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

enhance: Add json key inverted index in stats for optimization #38039

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open

enhance: Add json key inverted index in stats for optimization #38039

wants to merge 5 commits into from
+3,776 −217

Conversation

JsDove
Copy link
Contributor

@JsDove JsDove commented Nov 27, 2024

Add json key inverted index in stats for optimization
#36995

@sre-ci-robot sre-ci-robot added the size/XXL Denotes a PR that changes 1000+ lines. label Nov 27, 2024
@mergify mergify bot added the dco-passed DCO check passed. label Nov 27, 2024
@mergify Mergify
Copy link
Contributor

mergify bot commented Nov 27, 2024

@JsDove

Invalid PR Title Format Detected

Your PR submission does not adhere to our required standards. To ensure clarity and consistency, please meet the following criteria:

  1. Title Format: The PR title must begin with one of these prefixes:
  • feat: for introducing a new feature.
  • fix: for bug fixes.
  • enhance: for improvements to existing functionality.
  • test: for add tests to existing functionality.
  • doc: for modifying documentation.
  • auto: for the pull request from bot.
  1. Description Requirement: The PR must include a non-empty description, detailing the changes and their impact.

Required Title Structure:

[Type]: [Description of the PR]

Where Type is one of feat, fix, enhance, test or doc.

Example:

enhance: improve search performance significantly

Please review and update your PR to comply with these guidelines.

@JsDove JsDove changed the title Enhancement: Add json key inverted index in stats for optimization enhance: Add json key inverted index in stats for optimization Nov 27, 2024
@mergify mergify bot added kind/enhancement Issues or changes related to enhancement and removed do-not-merge/invalid-pr-format labels Nov 27, 2024
@mergify Mergify
Copy link
Contributor

mergify bot commented Nov 27, 2024

@JsDove cpp-unit-test check failed, comment rerun cpp-unit-test can trigger the job again.

@mergify Mergify
Copy link
Contributor

mergify bot commented Nov 27, 2024

@JsDove go-sdk check failed, comment rerun go-sdk can trigger the job again.

@mergify Mergify
Copy link
Contributor

mergify bot commented Nov 27, 2024

@JsDove E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

@mergify Mergify
Copy link
Contributor

mergify bot commented Nov 27, 2024

@JsDove go-sdk check failed, comment rerun go-sdk can trigger the job again.

@mergify Mergify
Copy link
Contributor

mergify bot commented Dec 2, 2024

@JsDove cpp-unit-test check failed, comment rerun cpp-unit-test can trigger the job again.

@mergify Mergify
Copy link
Contributor

mergify bot commented Dec 2, 2024

@JsDove E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

1 similar comment
@mergify Mergify
Copy link
Contributor

mergify bot commented Dec 3, 2024

@JsDove E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

@mergify Mergify
Copy link
Contributor

mergify bot commented Dec 3, 2024

@JsDove go-sdk check failed, comment rerun go-sdk can trigger the job again.

@mergify Mergify
Copy link
Contributor

mergify bot commented Dec 3, 2024

@JsDove cpp-unit-test check failed, comment rerun cpp-unit-test can trigger the job again.

@mergify Mergify
Copy link
Contributor

mergify bot commented Dec 10, 2024

@JsDove E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

@mergify Mergify
Copy link
Contributor

mergify bot commented Dec 10, 2024

@JsDove go-sdk check failed, comment rerun go-sdk can trigger the job again.

@mergify Mergify
Copy link
Contributor

mergify bot commented Dec 10, 2024

@JsDove cpp-unit-test check failed, comment rerun cpp-unit-test can trigger the job again.

@mergify Mergify
Copy link
Contributor

mergify bot commented Dec 10, 2024

@JsDove E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

@JsDove
Copy link
Contributor Author

JsDove commented Dec 10, 2024

/run-cpu-e2e

@mergify Mergify
Copy link
Contributor

mergify bot commented Dec 10, 2024

@JsDove cpp-unit-test check failed, comment rerun cpp-unit-test can trigger the job again.

@czs007 czs007 mentioned this pull request Dec 11, 2024
Closed
@mergify Mergify
Copy link
Contributor

mergify bot commented Dec 13, 2024

@JsDove cpp-unit-test check failed, comment rerun cpp-unit-test can trigger the job again.

@mergify Mergify
Copy link
Contributor

mergify bot commented Jan 5, 2025

@JsDove go-sdk check failed, comment rerun go-sdk can trigger the job again.

@mergify Mergify
Copy link
Contributor

mergify bot commented Jan 5, 2025

@JsDove E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

@JsDove
Copy link
Contributor Author

JsDove commented Jan 6, 2025

rerun go-sdk

@mergify Mergify
Copy link
Contributor

mergify bot commented Jan 6, 2025

@JsDove go-sdk check failed, comment rerun go-sdk can trigger the job again.

@JsDove
Copy link
Contributor Author

JsDove commented Jan 6, 2025

rerun go-sdk

@mergify Mergify
Copy link
Contributor

mergify bot commented Jan 6, 2025

@JsDove go-sdk check failed, comment rerun go-sdk can trigger the job again.

@JsDove JsDove force-pushed the optimization_json branch from cb6a311 to 223a346 Compare January 6, 2025 11:41
@mergify Mergify
Copy link
Contributor

mergify bot commented Jan 6, 2025

@JsDove E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

zhagnlu
zhagnlu reviewed Jan 7, 2025
View reviewed changes
internal/core/src/exec/expression/TermExpr.cpp Outdated Show resolved Hide resolved
internal/core/src/index/JsonKeyInvertedIndex.cpp Outdated Show resolved Hide resolved
internal/core/src/index/JsonKeyInvertedIndex.cpp Outdated Show resolved Hide resolved
internal/indexnode/task_stats.go Outdated Show resolved Hide resolved
@mergify Mergify
Copy link
Contributor

mergify bot commented Jan 7, 2025

@JsDove E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

@mergify Mergify
Copy link
Contributor

mergify bot commented Jan 8, 2025

@JsDove go-sdk check failed, comment rerun go-sdk can trigger the job again.

@mergify Mergify
Copy link
Contributor

mergify bot commented Jan 8, 2025

@JsDove E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

@JsDove JsDove force-pushed the optimization_json branch from 21a428b to 46d0809 Compare January 8, 2025 11:58
@mergify Mergify
Copy link
Contributor

mergify bot commented Jan 8, 2025

@JsDove go-sdk check failed, comment rerun go-sdk can trigger the job again.

@JsDove
Copy link
Contributor Author

JsDove commented Jan 8, 2025

rerun go-sdk

@mergify Mergify
Copy link
Contributor

mergify bot commented Jan 8, 2025

@JsDove E2e jenkins job failed, comment /run-cpu-e2e can trigger the job again.

luzhang and others added 4 commits January 9, 2025 10:14
@JsDove
add json key inverted index in stats to speed up json expr
dcf832b 
Signed-off-by: luzhang <[email protected]>
Co-authored-by: luzhang <[email protected]>
@JsDove
jsoncontainexpr unaryexpr binaryexpr json index optimization
4a6b820 
Signed-off-by: Xianhui.Lin <[email protected]>

improve jsonContainSExpr

Signed-off-by: Xianhui.Lin <[email protected]>

add jsonindex test

Signed-off-by: Xianhui.Lin <[email protected]>
@JsDove
support load json index after loadsegment
2a9aba2 
Signed-off-by: Xianhui.Lin <[email protected]>

improve statschecker unittest

Signed-off-by: Xianhui.Lin <[email protected]>

jsonindex expr code format

Signed-off-by: Xianhui.Lin <[email protected]>

fix go format

Signed-off-by: Xianhui.Lin <[email protected]>

fix controllerbasetest fail

Signed-off-by: Xianhui.Lin <[email protected]>

fix jsonindex memeroy leak

Signed-off-by: Xianhui.Lin <[email protected]>

fix jsonkey go format

Signed-off-by: Xianhui.Lin <[email protected]>

fix jsonindex go codeformat

Signed-off-by: Xianhui.Lin <[email protected]>

improve jsoninvert unitest

Signed-off-by: Xianhui.Lin <[email protected]>

delete unuse code

Signed-off-by: Xianhui.Lin <[email protected]>

refine test_json_key_index

Signed-off-by: Xianhui.Lin <[email protected]>

fix cpp unitest

Signed-off-by: Xianhui.Lin <[email protected]>

delete loginfo

Signed-off-by: Xianhui.Lin <[email protected]>

fix complie error

Signed-off-by: Xianhui.Lin <[email protected]>

fix codeformat

Signed-off-by: Xianhui.Lin <[email protected]>

fix createindex again hang up

Signed-off-by: Xianhui.Lin <[email protected]>

fix unitest

Signed-off-by: Xianhui.Lin <[email protected]>

fix createindex hang

Signed-off-by: Xianhui.Lin <[email protected]>

fix triggerstatstask go ut

Signed-off-by: Xianhui.Lin <[email protected]>

fix jsonindex filter error

Signed-off-by: Xianhui.Lin <[email protected]>

fix jsonindex filter error

Signed-off-by: Xianhui.Lin <[email protected]>

fix format

Signed-off-by: Xianhui.Lin <[email protected]>

improve jsonkey unitest

Signed-off-by: Xianhui.Lin <[email protected]>

remove sealsegment chunknum assert

Signed-off-by: Xianhui.Lin <[email protected]>
@JsDove
fix codereview
0a18c04 
Signed-off-by: Xianhui.Lin <[email protected]>
@JsDove JsDove force-pushed the optimization_json branch from 46d0809 to a5d5f1a Compare January 9, 2025 02:14
@mergify Mergify
Copy link
Contributor

mergify bot commented Jan 9, 2025

@JsDove go-sdk check failed, comment rerun go-sdk can trigger the job again.

@JsDove JsDove force-pushed the optimization_json branch from a5d5f1a to 1058d0b Compare January 9, 2025 03:43
@mergify Mergify
Copy link
Contributor

mergify bot commented Jan 9, 2025

@JsDove go-sdk check failed, comment rerun go-sdk can trigger the job again.

@JsDove JsDove force-pushed the optimization_json branch 2 times, most recently from a23a2cf to d4c04ab Compare January 9, 2025 06:39
@JsDove
fix jsonindex parse string to int
1852709 
Signed-off-by: Xianhui.Lin <[email protected]>

fix unitest

Signed-off-by: Xianhui.Lin <[email protected]>

improve

Signed-off-by: Xianhui.Lin <[email protected]>

fix

Signed-off-by: Xianhui.Lin <[email protected]>

fix

Signed-off-by: Xianhui.Lin <[email protected]>

fix go test error

Signed-off-by: Xianhui.Lin <[email protected]>
@JsDove JsDove force-pushed the optimization_json branch from d4c04ab to 1852709 Compare January 9, 2025 07:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zhagnlu zhagnlu zhagnlu left review comments

@bigsheeper bigsheeper Awaiting requested review from bigsheeper

@cydrain cydrain Awaiting requested review from cydrain

Assignees
No one assigned
Labels
area/compilation area/internal-api dco-passed DCO check passed. kind/enhancement Issues or changes related to enhancement size/XXL Denotes a PR that changes 1000+ lines.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@JsDove @sre-ci-robot @zhagnlu