[plugin] add option to compile with the Static Linux SDK instead of docker

[sebsto]

Expected behavior

Now that the Static Linux SDK has been released, it open the doors to cross-platform compilation from Mac that doesn't require Docker or depend on Amazon Linux.
The binaries created with the Static Linux SDK work "as is" on AWS Lambda.

I propose to add an option to the Archiver plugin to compile the binary with the Static Linux SDK instead of Docker.

When this option is enabled (it might become the default at some stage):

1. the plugin would check if the Linux Static SDK is installed, and install the correct version automatically when it's not installed.
2. the plugin will check the current platform architecture (or a command line flag to force a specific architecture)
3. the plugin will then compile the Lambda functions with swift build -c release --swift-sdk aarch64-swift-linux-musl (or x64_86)
4. the plugin will create a bootstrap sym link to the executable and package the sym link and the executable in a ZIP file

However, some pre-requisites are necessary before to release this enhancement

• Swift 6 must be generally available. At the moment, installing the toolchain requires manual steps that we don't want to include in the archive plugin code
• The Swift runtime for AWS Lambda must compile with musl (see Allow to compile with the Linux Static SDK (aka musl) #331)
• The segmentation fault problem must be fixed, to avoid further ceremony to get the toolchain working.

Actual behavior

The archiver plugin requires Docker and an Amazon Linux 2 image with the Swift toolchain installed

Steps to reproduce

n/a

If possible, minimal yet complete reproducer code (or URL to code)

Here is the script I use to test this on macOS Sonoma

export TOOLCHAINS=org.swift.600202406131a
PATH_TO_TOOLCHAIN=/Library/Developer/Toolchains/swift-6.0-DEVELOPMENT-SNAPSHOT-2024-06-13-a.xctoolchain

LAMBDA_USE_LOCAL_DEPS=true DYLD_LIBRARY_PATH=$PATH_TO_TOOLCHAIN/usr/lib/swift/macosx $PATH_TO_TOOLCHAIN/usr/bin/swift build -c release --swift-sdk aarch64-swift-linux-musl

pushd .build/aarch64-swift-linux-musl/release
cp UrlLambda bootstrap 
zip UrlLambda.zip bootstrap 
cp SQSLambda bootstrap 
zip SQSLambda.zip bootstrap 
cp HttpApiLambda bootstrap 
zip HttpApiLambda.zip bootstrap 
rm bootstrap
popd

What version of this project (swift-aws-lambda-runtime) are you using?

1.0.0-alpha1

Swift version

 ✗ swift --version && uname -a 
Apple Swift version 6.0-dev (LLVM 6e13b8be259f20f, Swift 372de4670b9cd55)
Target: arm64-apple-macosx14.0
Darwin bcd07458a91c 23.5.0 Darwin Kernel Version 23.5.0: Wed May  1 20:12:58 PDT 2024; root:xnu-10063.121.3~5/RELEASE_ARM64_T6000 arm64

Amazon Linux 2 docker image version

No response

[sebsto]

Discussion thread on the Swift forums : https://forums.swift.org/t/aws-lambda-functions-and-the-linux-static-sdk-works/72604

[fabianfett]

Are we aware that this will increase code size significantly?

[sebsto]

I am. I want to first measure the impact on code size and cold startup time. I want to share result with the community and - if it makes sense - propose it as an optional flag.

Unfortunately, I don't expect having time to work on this this year