Try to find pnpm-lock.yaml file upwards on tree structure

[Yurickh]

What are you trying to accomplish?

Iterate the folder tree upwards in order to find a pnpm-lock file when the directory source points to a nested workspace.

This hopefully addresses #10758, but I'd like to be able to verify these changes against my own repository. Is there any way to point dependabot to a fork in GitHub? I couldn't find any docs on it.

Anything you want to highlight for special attention from reviewers?

I'm not familiar with ruby project standards, so please let me know if you'd rather structure tests or code in a different way. I tried following the existing standards, but might have missed something.

How will you know you've accomplished your goal?

• The test I've added was failing prior to the changes in code, which I hope is enough to reproduce the issue described in Dependabot doesn't work with monorepos using pnpm #10758.
• I'd like to test these changes against my own repository, but couldn't find a way to point dependabot to a fork.

Checklist

• I have run the complete test suite to ensure all tests and linters pass.
• I have thoroughly tested my code changes to ensure they work as expected, including adding additional tests for new functionality.
• I have written clear and descriptive commit messages.
• I have provided a detailed description of the changes in the pull request, including the problem it addresses, how it fixes the problem, and any relevant details about the implementation.
• I have ensured that the code is well-documented and easy to understand.

[sylver]

Having this specific issue with our repos and was surprised to discover this is not actually covered while pnpm is supported.

Would be great to have some maintainers attention, since this is mandatory for any monorepo out there and the PR is quite simple and straightforward

[abdulapopoola]

Thanks @Yurickh , @thavaahariharangit will be investigating in this area this week.

[thavaahariharangit]

How do you ensure this solution works? , Is there are any example repository to test this scenario? , I mean I could recreate the error, and check that changes that you have provided in the PR fixing the issue? please.

Based on the analysis on the ticket. If it is yarn monorepo, dependabot is failing. But I could not test that on the repo they have provided due to the dependencies using private registries. Could you please let me know your thoughts on this.

[Yurickh]

That's my question in the PR description exactly :)~
Is there a way to run dependabot against a fork? I'd gladly setup a reproduction repo if I knew how to, but I couldn't find any docs on it.

[Yurickh]

So far, I've ensured the fix works by adding a test that breaks on your main, and passes on my branch, but if that's not enough, please I'd be glad to help verify it in any manner you see fit.

[thavaahariharangit]

Give me sometime, I will recreate this problem on sample or general repo, and verify both pls

1. Does that gives the same error customer is facing
2. Fix that you given here, fixing that issue.

[thavaahariharangit]

Existing Yarn fixes added to the PNPM as well.

[thavaahariharangit]

@Yurickh

I did some more reading on this issue. As per the documentation.
pnpm-lock.yaml should be on the root folder. If that's the case then this changes need not to be there.

FYR: #6346 (comment)

[Yurickh]

Yes, you're right there's only a single pnpm-lock.yaml file, but it is not necessarily at the same directory as the updates.directory configuration in dependabot.
In our case, we specifically want a single PR per workspace package, so we separate the configuration in different updates groups, but then the lockfile is not updated correctly.
This change ensures that we can find the root even when updates.directory is not the root by itself.

[thavaahariharangit]

Yes, you're right there's only a single pnpm-lock.yaml file, but it is not necessarily at the same directory as the updates.directory configuration in dependabot. In our case, we specifically want a single PR per workspace package, so we separate the configuration in different updates groups, but then the lockfile is not updated correctly. This change ensures that we can find the root even when updates.directory is not the root by itself.

Thank you for this update, now I understood the problem, Trying to recreate this problem based on your comment. Give me sometime pls :)

[MattIPv4]

If it helps for reproducing, I just ran into this bug/failure mode in the wild: alveusgg/alveusgg#856

[thavaahariharangit]

@Yurickh and @MattIPv4

I have created a sample repo to recreate this problem

Note:
In above repo I have used below sample dependencies,

1. root level: [email protected]
2. project level: [email protected]

Without any updates to the current dependabot (without this PR changes), I saw below behaviors

Scenario 1

When updates.directory is

directories:
      - /
      - /packages/*

then PR's generated

 +---------------------------------------------+
 |     Changes to Dependabot Pull Requests     |
 +---------+-----------------------------------+
 | created | express ( from 4.10.0 to 4.21.1 ) |
 | created | vue ( from 3.2.0 to 3.5.13 )      |
 +---------+-----------------------------------+

Scenario 2

When updates.directory is

directories:
      - /

then PR's generated

 +---------------------------------------------+
 |     Changes to Dependabot Pull Requests     |
 +---------+-----------------------------------+
 | created | express ( from 4.10.0 to 4.21.1 ) |
 +---------+-----------------------------------+

Scenario 3

When updates.directory is

directories:
      - /packages/*

then PR's generated

 +----------------------------------------+
 |  Changes to Dependabot Pull Requests   |
 +---------+------------------------------+
 | created | vue ( from 3.2.0 to 3.5.13 ) |
 +---------+------------------------------+

As far as I know things are working as per the dependabot documentation.

Could you please help me on recreating the issue that we are planning to address by this PR changes.

FYI
@abdulapopoola and @landongrindheim

[sylver]

@thavaahariharangit as stated previously, we have the exact same problem, thus why I ended up here.
So I can confirm this is a real issue.

To help illustrate, here's (a simplified snippet of) our dependabot.yml:

updates:
  - package-ecosystem: npm
    directory: "/apps/portal"
    schedule:
      interval: daily
    open-pull-requests-limit: 2
    allow:
      - dependency-name: "@angular*"
      - dependency-name: "typescript"
    ignore:
      - dependency-name: "@angular*"
        versions: [">=18.0.0"]
      - dependency-name: "@angular*"
        update-types:
          - "version-update:semver-major"
      - dependency-name: "typescript"
        update-types:
          - "version-update:semver-minor"
          - "version-update:semver-major"
    groups:
      angular:
        patterns:
          - "@angular*"
        update-types:
          - "minor"
          - "patch"

  - package-ecosystem: npm
    directory: "/"
    schedule:
      interval: daily
    open-pull-requests-limit: 4
    ignore:
      # Handled by dedicated groups above
      - dependency-name: "@angular*"

• the first update is needed for specific versions to lock in the directory /apps/portal. Dependabot will update /apps/portal/package.json but not /pnpm-lock.yaml <= That's the problem
• the second will properly update any /**/package.json along with /pnpm-lock.yaml, as expected, since we don't filter in a sub-directory/package of the monorepo

Hope that helps.

[sylver]

@thavaahariharangit I just had a look at your sample repo. You have a pnpm-lock.yaml in the subdirectory packages/web, but that's not how it's supposed to work. In a monorepo, everything is handled by one pnpm-lock.yaml file at the root level, not within each workspace packages.

[Yurickh]

Here's a reproduction repro:
https://github.com/Yurickh/dependabot-core-repro

I've created a very simple pnpm workspaces repository with two packages with old dependencies, and as you can see in the PRs opened:

• https://github.com/Yurickh/dependabot-core-repro/pull/3/files
• https://github.com/Yurickh/dependabot-core-repro/pull/2/files
• https://github.com/Yurickh/dependabot-core-repro/pull/1/files

None of them updates the lock file, as you'd expect.

[thavaahariharangit]

@thavaahariharangit I just had a look at your sample repo. You have a pnpm-lock.yaml in the subdirectory packages/web, but that's not how it's supposed to work. In a monorepo, everything is handled by one pnpm-lock.yaml file at the root level, not within each workspace packages.

Yes you're correct @sylver , But in this specific @Yurickh case it's different, Please refer #10806 (comment)

I might need to analyze your issue as a different ticket. First I will finish with @Yurickh's issue and get back to you.

[thavaahariharangit]

Here's a reproduction repro: https://github.com/Yurickh/dependabot-core-repro

I've created a very simple pnpm workspaces repository with two packages with old dependencies, and as you can see in the PRs opened:

• https://github.com/Yurickh/dependabot-core-repro/pull/3/files
• https://github.com/Yurickh/dependabot-core-repro/pull/2/files
• https://github.com/Yurickh/dependabot-core-repro/pull/1/files

None of them updates the lock file, as you'd expect.

I started analyzing your Issue, and I will get back to you as soon as I found anything.

[sylver]

Yes you're correct @sylver , But in this specific @Yurickh case it's different, Please refer #10806 (comment)

@thavaahariharangit I could be mistaken, but unless @Yurickh proves me wrong, I'm pretty sure we are talking about the same thing here.

[Yurickh]

As far as I can tell @sylver has indeed the same issue this PR is aiming to fix.

[sylver]

As far as I can tell @sylver has indeed the same issue this PR is aiming to fix.

Thanks @Yurickh

Now that this is out of the way, and given the length we took to explain the (simple) problem into details (while the changes in the PR are straightforward and already self-explanatory), what is preventing @thavaahariharangit to actually understand the problem and move forward ? @abdulapopoola

[thavaahariharangit]

Part of my analysis.

I have ran the cli against sample repo provided by @Yurickh

• Root level I don't see any different in PR generation with and without this changes. As expected as it is traversing upwards

• In Package-a level I notice the below different in PR generation with and without this changes

• In Package-b level I found an error That's what I took some extra time on this analysis

As I have decided analysis this error as part of another ticket. I am approving this, Moving forward and deploying this changes.

[sylver]

Thanks @thavaahariharangit, that will help a lot 🙏
Looking forward to the next release so that we can actually benefit from it

[sylver]

@thavaahariharangit after new PRs opened by dependabot in our repos, it seems it still does not update the pnpm-lock.yaml file from sub-packages rules even with this PR merged, not sure why though.

The issue is still like I described here earlier and like it was shown in @Yurickh monorepo sample example opened PRs:
Yurickh/dependabot-core-repro#3
Yurickh/dependabot-core-repro#2

@Yurickh could you force dependabot to update/reopen the PRs in your sample monorepo, so that we can assess whether it works or not at least on that one for a follow-up please ?

[Yurickh]

Yes, it seems the issue is still not fixed :(

[thavaahariharangit]

@sylver or @Yurickh Could you please raise a ticket on this please, I am able to prioritize this on upcoming planning meeting.

[MattIPv4]

Hiya, I just wanted to note that this fix does appear to be working, dependant successfully opened PRs in alveusgg/alveusgg finally

[MattIPv4]

I think the issue with the repro repo is that it is pointing dependabot to each of the sub-packages individually.

I played around a bunch in alveusgg/alveusgg and found that with this fix in, it all worked perfectly once we just pointed dependabot at the root -- it would discover and update the sub-packages by itself

[abdulapopoola]

I think the issue with the repro repo is that it is pointing dependabot to each of the sub-packages individually.

I played around a bunch in alveusgg/alveusgg and found that with this fix in, it all worked perfectly once we just pointed dependabot at the root -- it would discover and update the sub-packages by itself

Thanks @MattIPv4 ! @Yurickh could you give this a try?

@thavaahariharangit , can we start a discussion post with this guide too?

[sylver]

I think the issue with the repro repo is that it is pointing dependabot to each of the sub-packages individually.

Which is the point and why we need a fix and why the sample repo is highlighting that (valid) use case.

[MattIPv4]

I'm not sure I follow -- if you point dependabot at just the root, it will handle updating the entire workspace?

[sylver]

I'm not sure I follow -- if you point dependabot at just the root, it will handle updating the entire workspace?

But you have use cases in a monorepo where you need to filter your rules per packages / sub-folders, not at the root level

[MattIPv4]

Ah that is true. This fix seems to have solved the issue for it updating the sub-packages when pointed at the root, but not pointed directly at the sub-packages. I guess another issue needs to be filed?

[vluoto]

Ah that is true. This fix seems to have solved the issue for it updating the sub-packages when pointed at the root, but not pointed directly at the sub-packages. I guess another issue needs to be filed?

I filed a new issue for this: #11135