Enable usage of modules from runtime image in layers

[gunnarmorling]

Hey @aalmiray, this just came up on Reddit: Can we load modules from a modular runtime image (jlink) into a layered Layrry app? I think this will require some work, as the notition of GAV fails flat there.

[aalmiray]

Indeed. I suppose --layers-config and --properties make no sense (or do they?)
I'd expect all required modules to be present in the runtime image, in which case the layers configuration should mention module ids instead of GAVs. This poses the problem with duplicate & incompatible modules, you wouldn't be able to create an image with that setup, would you?
And what of dynamic plugins? do we expect them to be also available in the image? Doesn't make much sense, some could be in the image, others could be external.

[gunnarmorling]

Yes, this all is rather blurry to me. I suppose you could only have one version of a module in a modular runtime image. Re dynamic plug-ins, yeah they they'd not be part of the image. I think the layers config file would still make sense, if you wanted to load modules from other layers on top of a base layer in the modular runtime image. Maybe it make sense to consider the (non-JDK) modules in the image as one kind of base layer, with potentially other layers on top of that. This all needs exploration really.

[aalmiray]

Agreed. Let's tackle this after 1.0.0.

[aalmiray]

If the modularized application has a custom launcher then it can be launched directly from the jlinked image, as no external config nor properties files may be needed. External plugins should still be possible in this configuration, right?

[gunnarmorling]

Yes, exactly. The key question to me is whether you could somehow indicate that the modules of the image should be loaded into multiple layers (if that's desirable)? And indeed the launcher should be part of the image in this case, instead of relying on a separate Layrry install.

External plugins should still be possible in this configuration, right?

Yes, right.

[aalmiray]

I guess, if the launcher class remains the entry point even inside the image, then no additional modules should be loaded until the Launcher has its says, shouldn't they?

[gunnarmorling]

Yes, I'd say so.

[aalmiray]

I'll check by re-adding a custom runner to modular-tiles ;-)

[gunnarmorling]

Yeah, so that'd be a nice experiment: run modular-tiles in a container image, with the launcher baked in; and perhaps one plug-in, too, and other plug-ins could be added externally. Would be interesting to see how that'd look like.

…

[aalmiray]

I think it's doable except for the internal plugin part. It cannot be defined in the same "plugins" layer as the external plugins, thus it becomes just another module of the "core" layer.

[gunnarmorling]

You could have multiple plug-in layers, though, one for that that "internal" plug-in (which couldn't be uninstalled), and one for the dynamic external ones.

…

[aalmiray]

Yes, indeed. I meant to say the internal plugin layer cannot have adirectory property but rather a modules list 😏 .

[gunnarmorling]

Ah yes, absolutely. There'd have to be a third way for defining a layer's contents (besides GAV and a plug-ins directory); probably just module names in this case.

[aalmiray]

In that case I'd guess it would be wise to /s/modules/artifacts in Layer to open the door for a module property, wouldn't you agree?

[gunnarmorling]

Yes, perhaps. Or we'd detect whether a value under modules is a GAV (in which case we'd obtain it via Maven) or a module id (in which case we'd expect it to be part of the module path).

[aalmiray]

the more I think about having the launcher embedded in the jlinked image I get more questions. I can see (at least) the following scenarios:

1. All required modules are found in the image. No dynamic plugins. No need for external configuration.
2. All required modules are found in the image. Dynamic plugins desired, directories are set relative to the image. No need for external configuration.
3. All required modules are found in the image. Dynamic plugins desired, directories may reside anywhere, paths must be supplied as run args, however no external configuration may be needed due to custom launcher accepting args and arranging Layers before booting the app.
4. Starting modules are found in the image. Additional modules may be pulled from repositories (such as JavaFX deps that target a specific platform). No need for external configuration but non-linked modules must be defined using GAV.
5. Starting modules are found in the image. Additional modules may be pulled from repositories. Dynamic plugins are desired, directories are set relative to the image. No external configuration needed.
6. Starting modules are found in the image. Additional modules may be pulled from repositories. Dynamic plugins desired, directories may reside anywhere, paths must be supplied as run args, however no external configuration may be needed due to custom launcher accepting args and arranging Layers before booting the app.
7. Starting modules are found in the image. Configurable additional modules pulled from repositories and external dynamic plugins. Requires custom launcher to setup "base" layers (from image) plus layers from external config.

The thing is, launching modules from within the image must be done in a way that the Layrry launcher "classpath" does not pollute the application classpath.

[gunnarmorling]

Great analysis! I think one core insight is that Layrry's own dependencies shouldn't be part of the boot layer but rather go into their own separate layer, so to not conflict with any modules/dependencies of the actual app. This might require some trickery, if we want to keep all of Layrry's code in the single bootstrap JAR.

[aalmiray]

I think #12 is needed before we can tackle this, as the core classes must be packaged in a module, preferably a full module.