AN-300 Fall back to large Docker image size if we can't find a real one #7673
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jgainerdewar
commented
Jan 3, 2025
| val userCommandImageSizeRoundedUpInGB = userCommandImageSizeInGB.ceil.toInt | ||
| .map(s => MemorySize(s.toDouble, MemoryUnit.Bytes).to(MemoryUnit.GB)) | ||
| .getOrElse(MemorySize(30, MemoryUnit.GB)) | ||
| val userCommandImageSizeRoundedUpInGB = userCommandImageSizeInGB.amount.ceil.toInt |
There was a problem hiding this comment.
Slight refactor because I didn't want to write .getOrElse(30000000000L)
aednichols
approved these changes
Jan 6, 2025
| val userCommandImageSizeInBytes = createPipelineParameters.jobDescriptor.dockerSize | ||
|
|
||
| // Compute the decompressed size based on the information available. If we couldn't get the image size, | ||
| // default to 30GB. Defaulting to 0 can cause task to run out of disk. (more in AN-300) |
There was a problem hiding this comment.
This is 30 GB extra for the image, right? Not a 30 GB disk overall.
| .map(s => MemorySize(s.toDouble, MemoryUnit.Bytes).to(MemoryUnit.GB)) | ||
| val (userCommandImageSizeInGB, userImageLogString) = maybeUserCommandImageSizeInGB match { | ||
| case Some(imageSize) => (imageSize, "user command image") | ||
| case None => (MemorySize(30, MemoryUnit.GB), "failed to obtain user command image size, using safe default") |
There was a problem hiding this comment.
I appreciate this specific message.
aednichols
approved these changes
Jan 6, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
See discussion in AN-300. This should fix cases where we can't find the Docker image size, so we don't bump the boot disk size to account for it, and tasks fail with out-of-disk during
docker pull.Batch will be handled in a different ticket.
Includes a revert of #7585, since this is a better-targeted fix for the same issue.
Release Notes Confirmation
CHANGELOG.mdCHANGELOG.mdin this PRCHANGELOG.mdbecause it doesn't impact community usersTerra Release Notes