How to use swift-package-dependencies-check with dependencies in private repos?

[mman]

Hi Marco,

I have many packages that use private SPM dependencies via github.

Common method to allow github workflows to support private dependencies in any form (could be submodule, could be private package via [email protected]:username/repo) is to use old school trick of mapping ssh deploy keys and virtual ssh hostnames along with .gitconfig url...insteadOf to your private repositories.

This can be automated via great action from shaunco: https://github.com/shaunco/ssh-agent/tree/git-repo-mapping.

Here is an example swift build && swift test workflow that uses private swift dependencies that I am using:

name: Swift Build & Test

on: [workflow_dispatch, push]

jobs:
  swift:
    name: Swift ${{ matrix.swift }} on ${{ matrix.os }}
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        os: [ubuntu-latest, macos-latest]
        swift: ["5.5", "5.6"]
    steps:
      - uses: shaunco/ssh-agent@git-repo-mapping
        with:
          ssh-private-key: |
            ${{ secrets.REPO1_SSH_PRIVATE_KEY}}
            ${{ secrets.REPO2_SSH_PRIVATE_KEY}}
          repo-mappings: |
            github.com/mman/repo1.git
            github.com/mman/repo2.git
      - uses: fwal/[email protected]
        with:
          swift-version: ${{ matrix.swift }}
      - uses: actions/checkout@v3
        with:
          submodules: recursive
      - name: Build
        run: swift build
      - name: Test
        run: swift test

I have tried to use the same approach with your action, but it does not work properly. I have also tried with https://github.com/getsidetrack/action-xcodeproj-spm-update and there it works nicely.

So I started investigating what is the issue and I think I have found it:

The shaunco/ssh-agent@git-repo-mapping will populate ~/.gitconfig and ~/.ssh/* of the workflow job with private keys and hostname aliases for all the private dependencies that you may have. The ~/.ssh/* private keys are then added to the workflow internal ssh-agent and used by the subsequent steps. So for example xcodebuild used by the https://github.com/getsidetrack/action-xcodeproj-spm-update will pick up the new config nicely and will happily check dependencies and create a PR.

But your action will fail because it uses a step to invoke swift-release-notes via nested docker and the ~/.ssh/config and ~/.gitconfig are not properly passed to the docker step.

I have been able to overcome this limitation by somehow (hard copy, volume mount in case of docker build) pushing the ~/.ssh and ~/.gitconfig to the docker step so that it can properly access the private dependencies, but I have not found an easy way to do this with your workflow without forking and modifying it heavily.

I am not necessarily suggesting I know how to fix this properly, but I just want the issue to exist here for anybody hitting the same limitation.

I will probably try to work around this limitation by skipping the swift-release-notes binary invocation, and by simply invoking swift package update directly and comparing the md5sum before/after the same way action-xcodeproj-spm-update does it.

Another option could be to build swift-release-notes directly inside the job, and thus avoiding jumping to another nested docker step, which will help inherit the job environment.

Thoughts?

[MarcoEidinger]

Hi Martin (@mman),

thank for your interest in this GitHub action and for the provided feedback!

I am wondering if your use case even works with an older version of this GitHub action which does not make use of swift-release-notes, e.g. 1.1.0. ?

swift-release-notes is built within this Docker-based action. Allowing the GitHub action and the example workflow to run on a Linux virtual environment was a good idea. At least so I thought. It's cheaper for the users. Running a GitHub workflow on a macOS virtual environment is 10x pricier than using a Linux virtual environment. I am using Docker to ensure a certain Swift version other than the Swift version which may or may not be installed in the Linux virtual environment.

I was not aware of the challenges regarding private repositories. A major difference to action https://github.com/getsidetrack/action-xcodeproj-spm-update is that it does not use Docker but a bash script and therefore rely on the Swift version used in the virtual environment..

If you can verify that an older version of this Docker-based action works for you when using swift package update that would be great. Then we can know if it is an option to introduce an input parameter to control which tool to be used internally (swift-release-notes or swift package update).

[mman]

Hi @MarcoEidinger

The 1.1.0 does not work either with private repositories, since it also uses nested docker step. So any special git/ssh configuration performed in the github job itself is lost and not pushed to the nested docker step.

I am currently using fwal/[email protected] action (see my original comment above) to configure swift version to be used in the github job for swift build && swift test without relying on the nested docker, so that may be a way for the future.

I will have time next week to fork and look at what can be done.

[MarcoEidinger]

Hi @mman ,

next week I will also look into using a composite action which I believe did not exist as an action option when I created this GitHub action. In the past I remember there were only Docker or Javascript based GitHub action.

But I have my concerns as the user might have to use a specific virtual environment. For example action-xcodeproj-spm-update should fail when running on a Linux environment as it needs xcodebuild