Pytorch and Poetry doesn't work together

[jphme]

• Poetry version: 1.6.1
• Python version: 3.9.18
• OS version and name: Ubuntu 22.04 / Docker
• pyproject.toml: see below

• I am on the latest stable Poetry version, installed using a recommended method.
• I have searched the issues of this repo and believe that this is not a duplicate.
• I have consulted the FAQ and blog for any relevant entries or release notes.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

Commented in #4231 but decided to create a new issue because the old one is closed.

Pytorch and Poetry is still a major problem and blocker for my usecase.

I use a Docker container with a Torch/Cuda env that shouldnt be touched and use poetry config virtualenvs.create false for deployment of poetry-based projects into that container.

If I specify torch = { version = "=2.0.1+cu118", source = "cuda" } with the correct source, poetry will re-download Torch for Python 3.10 (even if Python 3.9. is explicitly specified in the toml and there is no Python 3.10 installed) and if I add torch = { url = "https://download.pytorch.org/whl/cu118/torch-2.0.1%2Bcu118-cp39-cp39-linux_x86_64.whl"}, Poetry will also redownload Torch instead of using the currently installed vesion.

Is there any solution to make poetry aware of the pre-installed version? (I need to fix the torch version in the pyproject.toml file to prevent other libraries from overwriting it - with pip this works fine. It is not viable for our usecase to redownload/reinstall Pytorch everytime some project library is updated...

Thanks!

EDIT:

This problem could also be due to the use of poetry config virtualenvs.create false and poetry not discovering the correct system python Path, even if it is correctly set. Is there any way to specify the correct path to poetry?
As I said, if I export my requirements (with poetry export) to requirements.txt and install with pip, everything works as expected, dependencies are installed and Pytorch is not overwritten.

[dimbleby]

probably any discussion here will duplicate #6409 or #8521 or both

[jphme]

probably any discussion here will duplicate #6409 or #8521 or both

The problem (for me at least) was/is that both are closely related which is not obvious at all: It is a very common usecase to use torch inside a docker image based on NVIDIA's base images with a working CUDA environment and seemingly the problem also described in #8521 causes the Pytorch problems (which seem to be "solved" as far as pure installation goes).

But would be fine for me to cose this and move discussion to these other issues; my problem was that I always searched for the Pytorch issues but did not find any solution because discussion there ist mostly on install (and not version pinning/overwriting) problems.

[dimbleby]

apparently no-one has anything new to say, and even if they did it would be better not to spread the conversation across lots of issues - please close in favour of one or both of those others

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.