loadLibrary order causes javagi.path to be ignored #216

New issue

Closed

opened 2025-04-30 18:16:47 +02:00 by JFronny · 3 comments

JFronny commented

2025-04-30 18:16:47 +02:00

(Migrated from github.com)

In GLib, libgobject is loaded before libglib, even though it depends on it.
As a consequence, the JVM tries to load libglib by itself from the system PATH (not java.library.path!), which fails, causing startup to fail.
This also isn't the only dependency of this kind.
It is possible to load these libraries manually ahead of javagi in this order, which lets the application start:

epoxy-0.dll
ffi-7.dll
fribidi-0.dll
intl-8.dll
jpeg-8.2.2.dll
pcre2-16-0.dll
pcre2-32-0.dll
pcre2-8-0.dll
pcre2-posix-3.dll
pixman-1-0.dll
sass.dll
tiff4.dll
z.dll
glib-2.0-0.dll
gmodule-2.0-0.dll
gobject-2.0-0.dll
graphene-1.0-0.dll
gthread-2.0-0.dll
harfbuzz.dll
png16-16.dll
cairo-2.dll
cairo-gobject-2.dll
cairo-script-interpreter-2.dll
gio-2.0-0.dll
girepository-1.0-1.dll
girepository-2.0-0.dll
harfbuzz-gobject.dll
harfbuzz-subset.dll
pango-1.0-0.dll
pangowin32-1.0-0.dll
gdk_pixbuf-2.0-0.dll
pangocairo-1.0-0.dll
gtk-4-1.dll
adwaita-1-0.dll

Generated via this code

        var set = Arrays.stream(System.getProperty("javagi.path").split(File.pathSeparator))
                .map(Path::of)
                .flatMap(Gain::list)
                .collect(Collectors.toCollection(LinkedHashSet::new));
        while (!set.isEmpty()) {
            var p = set.removeFirst();
            try {
                LibLoad.loadLibrary(p.getFileName().toString());
            } catch (RuntimeException re) {
                set.add(p);
                continue;
            }
            System.out.println(p.getFileName().toString());
        }

This really isn't an issue on Linux where this property probably isn't used by anyone.
On Windows however, apps probably want to ship their own GTK binaries (ideally packaged neatly in some maven dependency), which is made quite a bit more difficult by this fact.

In `GLib`, `libgobject` is loaded before `libglib`, even though it depends on it. As a consequence, the JVM tries to load `libglib` by itself from the system `PATH` (not `java.library.path`!), which fails, causing startup to fail. This also isn't the only dependency of this kind. It is possible to load these libraries manually ahead of javagi in this order, which lets the application start: ``` epoxy-0.dll ffi-7.dll fribidi-0.dll intl-8.dll jpeg-8.2.2.dll pcre2-16-0.dll pcre2-32-0.dll pcre2-8-0.dll pcre2-posix-3.dll pixman-1-0.dll sass.dll tiff4.dll z.dll glib-2.0-0.dll gmodule-2.0-0.dll gobject-2.0-0.dll graphene-1.0-0.dll gthread-2.0-0.dll harfbuzz.dll png16-16.dll cairo-2.dll cairo-gobject-2.dll cairo-script-interpreter-2.dll gio-2.0-0.dll girepository-1.0-1.dll girepository-2.0-0.dll harfbuzz-gobject.dll harfbuzz-subset.dll pango-1.0-0.dll pangowin32-1.0-0.dll gdk_pixbuf-2.0-0.dll pangocairo-1.0-0.dll gtk-4-1.dll adwaita-1-0.dll ``` <details> <summary>Generated via this code</summary> ```java var set = Arrays.stream(System.getProperty("javagi.path").split(File.pathSeparator)) .map(Path::of) .flatMap(Gain::list) .collect(Collectors.toCollection(LinkedHashSet::new)); while (!set.isEmpty()) { var p = set.removeFirst(); try { LibLoad.loadLibrary(p.getFileName().toString()); } catch (RuntimeException re) { set.add(p); continue; } System.out.println(p.getFileName().toString()); } ``` </details> This really isn't an issue on Linux where this property probably isn't used by anyone. On Windows however, apps probably want to ship their own GTK binaries (ideally packaged neatly in some maven dependency), which is made quite a bit more difficult by this fact.

jwharm commented

2025-04-30 20:59:38 +02:00

(Migrated from github.com)

We can reverse the ordering of libgobject and libglib in the GLib class, but I don't think it's feasible to load the other libraries (in the right order), because they aren't listed in the gir files. What would you propose? I could add a code example to the documentation to show how to manually load all libraries in the right order (based on your list)?

We can reverse the ordering of `libgobject` and `libglib` in the GLib class, but I don't think it's feasible to load the other libraries (in the right order), because they aren't listed in the gir files. What would you propose? I could add a code example to the documentation to show how to manually load all libraries in the right order (based on your list)?

JFronny commented

2025-04-30 22:38:50 +02:00

(Migrated from github.com)

I don't really have a good solution for this.
I have messed about a bit with an approach that covers correctly loading the GTK binaries as well as distributing them in a test project, but that is hardly ready for widespread use.
A self-extracting dependency like what I tried to do there is probably the easiest from the developer perspective and could be provided as an optional module using builds by the windows-gtk script.
However, an installed, production-ready application should really be installing the binaries directly from the installer.
My previous experiment abuses the fact that the executable directory is always part of the library path on Windows, but that required two copies of every library to bridge the gap between these different approaches to loading libraries.
While this gives us installer-managed libraries, it isn't really a good solution since it wastes space and doesn't cover development.
Since these requirements are contradictory, I'm almost at the point of thinking that library loading should be delegated to additional modules, with a suggested default module that effectively does what LibLoad is today (perhaps this one can even be shipped by default?), an additional module that extracts the GTK libraries like in gtk-repackaged and manually loads them using a known-good load order (this can be discovered mechanically as is done in gtk-repackaged), and a similar module that uses a known path to load the same libraries, so they can be shipped with the installer.
This would also leave open the option for user-provided library loading code, which is kind of needed on Windows to support the various ways an application could be installed.

This feels extremely over-engineered, but I honestly don't have a better idea.

I don't really have a good solution for this. I have messed about a bit with an approach that covers correctly loading the GTK binaries as well as distributing them in [a test project](https://github.com/JFronny/gtk-repackaged), but that is hardly ready for widespread use. A self-extracting dependency like what I tried to do there is probably the easiest from the developer perspective and could be provided as an optional module using builds by the windows-gtk script. However, an installed, production-ready application should really be installing the binaries directly from the installer. My [previous experiment](https://github.com/jfronny/javagi-multiplatform) abuses the fact that the executable directory is always part of the library path on Windows, but that required two copies of every library to bridge the gap between these different approaches to loading libraries. While this gives us installer-managed libraries, it isn't really a good solution since it wastes space and doesn't cover development. Since these requirements are contradictory, I'm almost at the point of thinking that library loading should be delegated to additional modules, with a suggested default module that effectively does what LibLoad is today (perhaps this one can even be shipped by default?), an additional module that extracts the GTK libraries like in `gtk-repackaged` and manually loads them using a known-good load order (this can be discovered mechanically as is done in gtk-repackaged), and a similar module that uses a known path to load the same libraries, so they can be shipped with the installer. This would also leave open the option for user-provided library loading code, which is kind of needed on Windows to support the various ways an application could be installed. This feels extremely over-engineered, but I honestly don't have a better idea.

JFronny commented

2025-05-01 17:36:10 +02:00

(Migrated from github.com)

I have had an idea for this. Give me a few days and I should have a PR that at least solves this issue (though it won't solve distribution completely)

👍 1