Add support for lazy nodejs installation

[RobertCraigie]

Hey! I'd love to be able to add this as a base dependency for my package which needs to interface with a Node-based CLI. However, I am wary of doing this as it adds ~150MB to the installation size which may be unnecessary if a user already has Node installed on their machine.

My current solution is to add this project as an extra which means users can easily install it if it would improve their experience, e.g.

pip install prisma[node]

This being optional means I need a solution to download Node if it isn't present on the user's machine, this led me to nodeenv which works for now but in an ideal world these would be handled by the same package.

I don't think this would make sense for the current setup but when this project is changed to use the nodejs package name (#9) I think it could make a lot of sense to "namespace" the binary wheels, e.g. you would now install the binary distribution like this:

pip install nodejs[bin]

In the "lazy" case it would essentially do the same thing that nodeenv does, downloading Node binaries at runtime. This could be handled transparently by both the CLI and the Python API with an explicit API as well, e.g.

# implicit downloading if not present
from nodejs import node

node.call(['script.js', 'arg1', ...], **kwargs)

# explicit API
import nodejs
from pathlib import Path

nodejs.download()
nodejs.download(cache_dir=Path.home() / '.cache' / 'nodejs-pypi')

[samwillis]

Interesting ideas! They are sort of along the same route I was being to think about. I'm interested to explore how the nodejs package could have multiple ways to make a specific version of node available to your venv. nodejs-bin could then remain the pypi nodejs python wheel distribution, with nodejs using it when required.

The nodejs package could script pip to install a version of nodejs-bin when the system version of nodejs doesn't match the required version: https://pip.pypa.io/en/latest/user_guide/#using-pip-from-your-program

subprocess.check_call([sys.executable, '-m', 'pip', 'install', 'nodejs-bin'])

We could also check if the user has nvm, and if so has the correct version of node available. But I'm cautious of having too many options and moving parts. I want this to be simple, as you said nodeenv is already there as an advanced option.

They tricky part is tying into setup.py/requirements.txt version range pinning. What I want users to be able to do is specify a version range and that is what they get. Making this work "lazily" needs some thought:

# requirements.txt
nodejs>=14.15.0,<16.2.0

It's "easy" to make a nodejs package for each released version that knows how to check if its own version is installed. But Ranges seems much more complex.

If we can't solve that, then I would be inclined to suggest that we shouldn't go the lazy route at all.

Your use case of needing node for you package, but not wanting the user to have to worry about how to install it is exactly the problem I'm trying to solve. There are so many projects using both Python on the backend, and Nodejs tooling for the front end, that building a unified venv that covers both makes the developers life easer.

[RobertCraigie]

It's "easy" to make a nodejs package for each released version that knows how to check if its own version is installed. But Ranges seems much more complex.

If we can't solve that, then I would be inclined to suggest that we shouldn't go the lazy route at all.

Ah I hadn't considered the complexities of version ranges.

I think it should definitely be doable but would require more effort.

Actually how were you thinking of letting users propagate the version range? That metadata wouldn't be easily accessible to the nodejs package. You could try and find a requirements.txt file but that wouldn't work any poetry users.

I think a good solution would be to make use of a pyproject.toml config, e.g.

[tool.nodejs]
version = ">=14.15.0,<16.2.0"

The nodejs package could script pip to install a version of nodejs-bin when the system version of nodejs doesn't match the required version:

Were you thinking this would just use whatever the pip executable points to?

If so, I would have concerns about this as I think users wouldn't be particularly happy about a package automatically installing another package, even if they're related. I don't have any information backing this point up so it may be a non-issue.

I think a better solution could be for nodejs to manage virtual environments and install nodejs-bin inside its own virtual environment as to not mess with the user's environment. This shouldn't add too much complexity, just requiring a python -m venv <path> call before installation.

This also has the advantage of potentially making supporting version ranges easier as it would make it easy for nodejs to pick and choose which version of nodejs-bin to use as it wouldn't be reliant on the user's environment.