Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add caching of downloaded actions to achieve parity with the tool cache functionality #3551

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Add caching of downloaded actions to achieve parity with the tool cache functionality #3551

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions src/Runner.Common/Constants.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -256,6 +256,7 @@ public static class Agent
public static readonly string ForcedActionsNodeVersion = "ACTIONS_RUNNER_FORCE_ACTIONS_NODE_VERSION";
public static readonly string PrintLogToStdout = "ACTIONS_RUNNER_PRINT_LOG_TO_STDOUT";
public static readonly string ActionArchiveCacheDirectory = "ACTIONS_RUNNER_ACTION_ARCHIVE_CACHE";
public static readonly string ActionArchiveExternalCachingEnabled = "ACTIONS_RUNNER_ACTION_ARCHIVE_EXTERNAL_CACHING_ENABLED";
}

public static class System
Expand Down
17 changes: 12 additions & 5 deletions src/Runner.Worker/ActionManager.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -795,16 +795,17 @@ private async Task DownloadRepositoryActionAsync(IExecutionContext executionCont
var useActionArchiveCache = false;
var hasActionArchiveCache = false;
var actionArchiveCacheDir = Environment.GetEnvironmentVariable(Constants.Variables.Agent.ActionArchiveCacheDirectory);
var externalCachingEnabled = Convert.ToBoolean(Environment.GetEnvironmentVariable(Constants.Variables.Agent.ActionArchiveExternalCachingEnabled) ?? "false");
#if OS_WINDOWS
var cacheArchiveFile = Path.Combine(actionArchiveCacheDir, downloadInfo.ResolvedNameWithOwner.Replace(Path.DirectorySeparatorChar, '_').Replace(Path.AltDirectorySeparatorChar, '_'), $"{downloadInfo.ResolvedSha}.zip");
#else
var cacheArchiveFile = Path.Combine(actionArchiveCacheDir, downloadInfo.ResolvedNameWithOwner.Replace(Path.DirectorySeparatorChar, '_').Replace(Path.AltDirectorySeparatorChar, '_'), $"{downloadInfo.ResolvedSha}.tar.gz");
#endif
if (!string.IsNullOrEmpty(actionArchiveCacheDir) &&
Directory.Exists(actionArchiveCacheDir))
{
hasActionArchiveCache = true;
Trace.Info($"Check if action archive '{downloadInfo.ResolvedNameWithOwner}@{downloadInfo.ResolvedSha}' already exists in cache directory '{actionArchiveCacheDir}'");
#if OS_WINDOWS
var cacheArchiveFile = Path.Combine(actionArchiveCacheDir, downloadInfo.ResolvedNameWithOwner.Replace(Path.DirectorySeparatorChar, '_').Replace(Path.AltDirectorySeparatorChar, '_'), $"{downloadInfo.ResolvedSha}.zip");
#else
var cacheArchiveFile = Path.Combine(actionArchiveCacheDir, downloadInfo.ResolvedNameWithOwner.Replace(Path.DirectorySeparatorChar, '_').Replace(Path.AltDirectorySeparatorChar, '_'), $"{downloadInfo.ResolvedSha}.tar.gz");
#endif
if (File.Exists(cacheArchiveFile))
{
try
Expand All @@ -830,6 +831,12 @@ private async Task DownloadRepositoryActionAsync(IExecutionContext executionCont
if (!useActionArchiveCache)
{
await DownloadRepositoryArchive(executionContext, link, downloadInfo.Authentication?.Token, archiveFile);
if (hasActionArchiveCache && externalCachingEnabled)
{
executionContext.Output($"Saving archive file to cache at '{cacheArchiveFile}'");
Directory.CreateDirectory(Path.GetDirectoryName(cacheArchiveFile));
File.Copy(archiveFile, cacheArchiveFile, true);
}
Comment on lines +834 to +839
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By adding the bit of code in the PR we are able to step into the middle of the actions download functionality and save the downloaded archive to the cache folder without any change to the rest of the logic flow. On subsequent runs, that archive is found and used by all runners as intended.

Further, we also attempted to mount the NFS as the temp folder, just as a “let’s try it and see what happens” kind of thing, knowing that we would be persisting far too much data. That also did not work.

According to our testing, a mounted NFS archive directory, regardless of “tool” or “action” cache HAS to exist in a directory outside of any of the directories created by the runner. This means that in order for a mounted persistent cache to work, the actions caching process HAS to function in the way that the tool cache functions e.g., persisting the downloaded archive to the provided cache directory.

}

var stagingDirectory = Path.Combine(tempDirectory, "_staging");
Expand Down