Add tunable buffer size for spt3g filtering_istream #194
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds a small (and perhaps temporary) patch to specify the buffer size to the
filtering_istream
stack. This is a quick update to fix slow data loading and #192 will be rebased after this is merged. Also added GSL to the wheel build dependencies.For testing on a compute node on perlmutter.nersc.gov, I used a single process and 8 threads. I then loaded one wafer (ufm_mv12) of satp3 observation
obs_1714550584_satp3_1111111
.I used the new
SO3G_FILESYSTEM_BUFFER
environment variable to try a variety of buffer sizes loading this same data from the CFS and scratch filesystems. Note that the frame files in this case are each ~200MB, so the largest buffer size tested is actually larger than the frame files. The smaller values of buffer size were essentially unusable on CFS.Here is a plot of the times:
Note that CFS access is faster from a login node than a compute node, and scratch access is faster from a compute node than a login node. Here is the data in the plot: