Skip to content

Commit

Permalink
Physical query plan + executors (#1881)
Browse files Browse the repository at this point in the history
  • Loading branch information
@joshua-spacetime
joshua-spacetime authored Oct 24, 2024
1 parent 2d8224a commit c840eda
Show file tree
Hide file tree
Showing 9 changed files with 761 additions and 1 deletion.
20 changes: 20 additions & 0 deletions Cargo.lock
View file

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 4 additions & 0 deletions Cargo.toml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,12 @@ members = [
"crates/core",
"crates/data-structures",
"crates/durability",
"crates/execution",
"crates/expr",
"crates/fs-utils",
"crates/lib",
"crates/metrics",
"crates/physical-plan",
"crates/primitives",
"crates/sats",
"crates/schema",
Expand Down Expand Up @@ -96,9 +98,11 @@ spacetimedb-commitlog = { path = "crates/commitlog", version = "1.0.0-rc1" }
spacetimedb-core = { path = "crates/core", version = "1.0.0-rc1" }
spacetimedb-data-structures = { path = "crates/data-structures", version = "1.0.0-rc1" }
spacetimedb-durability = { path = "crates/durability", version = "1.0.0-rc1" }
spacetimedb-execution = { path = "crates/execution", version = "1.0.0-rc1" }
spacetimedb-expr = { path = "crates/expr", version = "1.0.0-rc1" }
spacetimedb-lib = { path = "crates/lib", default-features = false, version = "1.0.0-rc1" }
spacetimedb-metrics = { path = "crates/metrics", version = "1.0.0-rc1" }
spacetimedb-physical-plan = { path = "crates/physical-plan", version = "1.0.0-rc1" }
spacetimedb-primitives = { path = "crates/primitives", version = "1.0.0-rc1" }
spacetimedb-sats = { path = "crates/sats", version = "1.0.0-rc1" }
spacetimedb-schema = { path = "crates/schema", version = "1.0.0-rc1" }
Expand Down
12 changes: 12 additions & 0 deletions crates/execution/Cargo.toml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
[package]
name = "spacetimedb-execution"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
license-file = "LICENSE"
description = "The SpacetimeDB query engine"

[dependencies]
spacetimedb-lib.workspace = true
spacetimedb-expr.workspace = true
spacetimedb-table.workspace = true
Loading

2 comments on commit c840eda

@github-actions
Copy link

@github-actions github-actions bot commented on c840eda Oct 24, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Benchmarking failed. Please check the workflow run for details.

@github-actions
Copy link

@github-actions github-actions bot commented on c840eda Oct 24, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Callgrind benchmark results

Callgrind Benchmark Report

These benchmarks were run using callgrind,
an instruction-level profiler. They allow comparisons between sqlite (sqlite), SpacetimeDB running through a module (stdb_module), and the underlying SpacetimeDB data storage engine (stdb_raw). Callgrind emulates a CPU to collect the below estimates.

Measurement changes larger than five percent are in bold.

In-memory benchmarks

callgrind: empty transaction

db total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw 6516 6516 0.00% 6550 6550 0.00%
sqlite 5579 5579 0.00% 6047 6047 0.00%

callgrind: filter

db schema indices count preload _column data_type total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str no_index 64 128 1 u64 76682 76682 0.00% 77012 77012 0.00%
stdb_raw u32_u64_str no_index 64 128 2 string 119180 119180 0.00% 119688 119580 0.09%
stdb_raw u32_u64_str btree_each_column 64 128 2 string 25231 25234 -0.01% 25635 25666 -0.12%
stdb_raw u32_u64_str btree_each_column 64 128 1 u64 24199 24199 0.00% 24517 24521 -0.02%
sqlite u32_u64_str no_index 64 128 2 string 144713 144695 0.01% 146203 146189 0.01%
sqlite u32_u64_str no_index 64 128 1 u64 124044 124044 0.00% 125300 125300 0.00%
sqlite u32_u64_str btree_each_column 64 128 1 u64 131361 131361 0.00% 132741 132749 -0.01%
sqlite u32_u64_str btree_each_column 64 128 2 string 134494 134494 0.00% 136092 136092 0.00%

callgrind: insert bulk

db schema indices count preload total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 64 128 905986 905885 0.01% 964378 933133 3.35%
stdb_raw u32_u64_str btree_each_column 64 128 1056884 1054531 0.22% 1121246 1087287 3.12%
sqlite u32_u64_str unique_0 64 128 398320 398320 0.00% 413144 413144 0.00%
sqlite u32_u64_str btree_each_column 64 128 983637 983637 0.00% 1019011 1019027 -0.00%

callgrind: iterate

db schema indices count total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 1024 153946 153946 0.00% 154012 154012 0.00%
stdb_raw u32_u64_str unique_0 64 16971 16971 0.00% 17025 17021 0.02%
sqlite u32_u64_str unique_0 1024 1067261 1067261 0.00% 1070667 1070667 0.00%
sqlite u32_u64_str unique_0 64 76201 76219 -0.02% 77223 77241 -0.02%

callgrind: serialize_product_value

count format total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
64 json 47528 47528 0.00% 50214 50214 0.00%
64 bsatn 25509 25509 0.00% 27821 27821 0.00%
16 bsatn 8200 8200 0.00% 9628 9628 0.00%
16 json 12188 12188 0.00% 14126 14126 0.00%

callgrind: update bulk

db schema indices count preload total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 1024 1024 21033835 21036519 -0.01% 21745879 21738025 0.04%
stdb_raw u32_u64_str unique_0 64 128 1318442 1318206 0.02% 1364750 1395222 -2.18%
sqlite u32_u64_str unique_0 1024 1024 1802128 1802128 0.00% 1811252 1811252 0.00%
sqlite u32_u64_str unique_0 64 128 128474 128474 0.00% 131286 131286 0.00%
On-disk benchmarks

callgrind: empty transaction

db total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw 6521 6521 0.00% 6555 6555 0.00%
sqlite 5621 5621 0.00% 6149 6149 0.00%

callgrind: filter

db schema indices count preload _column data_type total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str no_index 64 128 1 u64 76687 76687 0.00% 77017 77017 0.00%
stdb_raw u32_u64_str no_index 64 128 2 string 120274 119185 0.91% 120694 119677 0.85%
stdb_raw u32_u64_str btree_each_column 64 128 2 string 25253 25237 0.06% 25649 25629 0.08%
stdb_raw u32_u64_str btree_each_column 64 128 1 u64 24204 24204 0.00% 24522 24526 -0.02%
sqlite u32_u64_str no_index 64 128 1 u64 125965 125965 0.00% 127569 127565 0.00%
sqlite u32_u64_str no_index 64 128 2 string 146616 146616 0.00% 148370 148374 -0.00%
sqlite u32_u64_str btree_each_column 64 128 2 string 136616 136634 -0.01% 138620 138646 -0.02%
sqlite u32_u64_str btree_each_column 64 128 1 u64 133457 133457 0.00% 135211 135211 0.00%

callgrind: insert bulk

db schema indices count preload total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 64 128 855656 855460 0.02% 913182 912754 0.05%
stdb_raw u32_u64_str btree_each_column 64 128 1007247 1006400 0.08% 1069975 1068810 0.11%
sqlite u32_u64_str unique_0 64 128 415857 415857 0.00% 430099 430103 -0.00%
sqlite u32_u64_str btree_each_column 64 128 1021898 1021898 0.00% 1055392 1055396 -0.00%

callgrind: iterate

db schema indices count total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 1024 153951 153951 0.00% 154001 154001 0.00%
stdb_raw u32_u64_str unique_0 64 16976 16976 0.00% 17026 17026 0.00%
sqlite u32_u64_str unique_0 1024 1070341 1070323 0.00% 1074155 1074137 0.00%
sqlite u32_u64_str unique_0 64 77973 77973 0.00% 79215 79215 0.00%

callgrind: serialize_product_value

count format total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
64 json 47528 47528 0.00% 50214 50214 0.00%
64 bsatn 25509 25509 0.00% 27821 27821 0.00%
16 bsatn 8200 8200 0.00% 9628 9628 0.00%
16 json 12188 12188 0.00% 14126 14126 0.00%

callgrind: update bulk

db schema indices count preload total reads + writes old total reads + writes Δrw estimated cycles old estimated cycles Δcycles
stdb_raw u32_u64_str unique_0 1024 1024 19546511 19541710 0.02% 20268893 20267228 0.01%
stdb_raw u32_u64_str unique_0 64 128 1272130 1272258 -0.01% 1347640 1348238 -0.04%
sqlite u32_u64_str unique_0 1024 1024 1809689 1809689 0.00% 1818393 1818393 0.00%
sqlite u32_u64_str unique_0 64 128 132600 132600 0.00% 135572 135572 0.00%

Please sign in to comment.