This new job resulted in a package which allows to run GIMP on Windows
(as tested in a VM; at least it starts, I can create a new canvas and
paint). Of course I think this will need to be tweaked a little bit
more, as I'm sure we miss things here and there.
At the very least, even though I add the Python and Luajit binaries,
GIMP on Windows didn't find them. This will need to be investigated.
Also it looks like opening from a remote location may not work. Not sure
if this about a missing GIO module or maybe something which works
differently on Windows (I was not even able to drag'n drop from the
browser!). Anyway this needs to be looked at as well.
Note that gdk-pixbuf-query-loaders is apparently unneeded when GIMP is
built this way (unlike with our crossroad build).
All this to say that this is still an early attempt to full CI build for
Windows.
It doesn't invalidate the crossroad build, because cross-compilation
builds from Linux will always stay very important for Linux developers
to be able to easily fix Windows bugs too; yet the crossroad build has 2
major issues:
1. We haven't figured out yet how to run GObject Introspection tools for
cross-builds, so the crossroad builds are not full-featured (and this
is quite a major feature we are missing!).
2. Also I will want to run the installer in the CI at some point and the
one we use can only run on Windows itself AFAIK. We could try to run
it through Wine, but still anyway the point 1. is already quite a
blocker, let's do the simple thing.
Note that we will likely want to move to meson for this build, because
autotools is very slow on Windows. But as long as the few blocker meson
bugs are not fixed, let's stick to the slow yet good build.
Without this, the native Windows GIMP build fails in CI with:
> make[2]: *** No rule to make target '_install\include\gegl-0.4\gegl.h', needed by '../libgimpcolor/gimpadaptivesupersample.lo'. Stop.
Even though `gegl.h` is present (as checked through artifacts). It's
just so weird.
The build rules were highly inspired by other projects on GNOME's
Gitlab. All of them used to build with ccache. It worked fine for the
main build, but completely broke GObject Introspection build on both
babl and GEGL. And the worse thing is that meson was absolutely not
displaying the error, just saying it failed (even in verbose mode). A
lot of time wasted trying to debug.
Therefore let's get rid of ccache, but only for babl and GEGL. Keep it
for GIMP itself as it works fine there.
Other minor changes:
* Build from the build dir, rather than source. The other way around
works too, but I actually find commands simpler this way.
* Adding artifacts.
Ok so this is horrible, but this is the only way I found to get past
some errors. The build would break randomly (sometimes it would,
sometimes not, with the same gitlab-ci rules) because of existing files.
The errors would be the following when trying to install dependency
packages with pacman:
> mingw-w64-x86_64-glib2: /mingw64/bin/libglib-2.0-0.dll exists in filesystem
At first I thought it was a bug in Pacman or MSYS2, but on some runners,
I had such warnings at the very start of the runner (note: on some
runners, the warnings would not be displayed even though the final
conflicting files error would still happen at the end of the logs):
> warning: failed to remove mingw64/bin/libglib-2.0-0.dll: Invalid argument
It looks like it might be related to this Gitlab bug where a runner
would fail to clean its environment:
https://gitlab.com/gitlab-org/gitlab-runner/-/issues/1839
In any case, the only way I found so far is to manually remove the
conflicting files before installing the packages supposed to install
these. This is completely horrible (and I sure hope it won't come up
again with different files each time) but really the only workaround I
found so far (I think a real solution will have to come from Gitlab code
or from the GNOME admins, not sure what is the exact source of the
problem).
Just an initial test to get a hang of the thing, mostly inspired from
GTK gitlab-ci rules adapted to our build.
All in one job (deps, babl, GEGL, GIMP itself) for now, for simplicity
of debugging. We'll see later to break this into CI sub-jobs.
… to GIMP not text layer rendering in image
Patch was merged n Cairo today, wouhou!
https://gitlab.freedesktop.org/cairo/cairo/-/merge_requests/114
If not mistaken, the fix should then appear in Cairo 1.17.6 or 1.18.0
(whatever comes next). As we are obviously not going to bump the Cairo
dependency so early, let's just add the patch, at the very least to be
used for our official builds. Also this way, we won't forget about this
issue in the future when we need to bump Cairo.
This is a patch for issue #913, the infamous "non-existent floppy drive"
or other unreachable network or unmounted drives. Well at least it
*should* fix the issue, but GLib developers are hoping we could test. So
let's add this in there for our next Windows package.
Nowadays .rs is the extension for the Rust programming language files,
and it's confusing that GIMP is trying to associate with them.
A simple rename of existing .rs images to .ras will allow them to be
opened again.
Note by reviewer: ideally file association should not rely on filename
extension, and should be detected properly (i.e. file "magic"). This way
even extension clash would not be a problem (format would be detected
whatever the extension used). Unfortunately it's apparently not the case
on Windows.
Anyway since nowadays chances to see a Rust code file are likely much
higher than seeing a Sun Raster image file, let's just accept this patch
and drop association of `.rs` on Windows.
Adding 2 temporary patches for our builds:
- gexiv2-mr20.patch: to be applied over GExiv2 0.12.1. This patch is
already contributed upstream so it won't be necessary once GExiv2
0.12.2 is released.
- 0001-Issue-5863-do-not-raise-WARNINGs-on-Exiv2-unsupporte.patch: to be
applied over GIMP master branch itself. It depends on GExiv2 0.12.2 so
I cannot push it to master until we bump GExiv2 minimum requirement
(which can only happen when v0.12.2 is out then available in Debian
testing, as per our dependency rules).
For the time being, while waiting for upstream releases, all we can do
is wait and apply these patches on our own dev builds (because this bug
is annoying and was reported to us quite a few times already).
This should give a nice name to distribution archives so that they are
not all called `artifacts.zip`. Names will better describe their
contents (target OS or source and short commit hash, because for CI
builds, it's important to know which commit is being tested).
Also replace CI_COMMIT_REF_NAME in other artifact names by
CI_COMMIT_REF_SLUG. Otherwise if a branch has a slash (quite common in
branch names), only the part after the last slash is used for archive
naming.
Finally immediately exits from dependency build with error code (!= 0)
if `crossroad install` command failed.
… the CI.
There are 2 finale steps before finale binary distribution on Windows.
We must compile the GSettings XML schema files and register GdkPixbuf
loaders (for file format support in the GUI).
I used to provide a wrapper to be run inside Windows before first GIMP
run. Never did I realize that I can compile the distributed GSettings
schemas with the native `glib-compile-schemas` (works fine in my tests).
As for the GdkPixbuf loaders, we inspect DLL libraries, hence we do
require the target `gdk-pixbuf-query-loaders` which is unfortunately a
Windows executable. Yet it seems to work fine with Wine, so let's be
done with it in the CI instead of requiring manual steps from testers of
the CI builds. Then a few `sed` calls are enough to make the path in the
produced text file relative instead of absolute (which works fine, again
in my tests at least).
This means that I don't have to distribute the 2 binaries and the DLLs
they depend on anymore. Moreover let's remove the wrapper (but still
generate one which just calls GIMP so that we call it from the tree
root, where it's much less messy).
Note: I failed to install wine32 (32-bit Wine) on the Gitlab runner.
After following all instructions, I encountered weird errors. So
instead, I just make the win32-nightly job depend on win64-nightly and
copy `loaders.cache` from one to another, as it is a
platform-independent text file (as long as we provide the same GdkPixbuf
loaders on both of course, which we do).
The main purpose of these jobs is to only package the strict necessary
for a working GIMP under Windows, i.e. getting rid of all unnecessary
executables, and inspecting binary dependencies recursively to only
package used DLLs.
The dll_link.py script is taken from Siril codebase (see commit a86e82a8
on Siril repository, by FlorianBen). This was a very nice idea, and
makes for much smaller test archive (Siril is also GPLv3 so licensing is
ok for the reuse, also anyway it's just a small independent build
script).
Moreover having it as a separate job allows to have artifacts with only
the finale distribution (artifacts on the build job also have the build
directory and the whole prefix, which we want to keep in order to debug
when needed).
Hopefully I am not missing anything. Siril seems to package more, like
various gdk-pixbuf-*.exe, gspawn-*.exe and gdbus.exe. I am wondering if
these are actually necessary. I could run GIMP fine without these in
quick tests, but I guess I'll have to investigate a bit more to figure
this out. That's what nightly builds are for, after all, so hopefully
people will report if we miss some runtime dependencies.
This commit makes sure we can properly run the tests in a headless
environment, i.e. they don't mess with the user's X display or their
session bus. The latter is also needed for parallel tests as they fail
to simultaneously own the same name on the session bus.
Replaced the "xvfb-run" meson option with the "headless" option, which
is more intuitive (and also more correct, since we now also require
`dbus-run-session` to run the tests, not only `xvfb-run`).
Finally, note that we need a version of `xvfb-run` that supports the
`-d` (`--auto-display`) option. The problem with `--auto-servernum`
which is also regularly used, is that it doesn't shut down cleanly,
returning a non-zero exit code, wich makes the test fail.
Fixes https://gitlab.gnome.org/GNOME/gimp/-/issues/5078
libopenexr was installed, but pkg-config was failing because of missing
dependency:
```
$ x86_64-w64-mingw32-pkg-config --modversion OpenEXR
Package IlmBase was not found in the pkg-config search path.
Perhaps you should add the directory containing `IlmBase.pc'
to the PKG_CONFIG_PATH environment variable
Package 'IlmBase', required by 'OpenEXR', not found
```
Looks like there may be a dependency bug in the openexr package in
Msys2. Anyway let's just add ilmbase and get this to be detected
correctly.
Our installer use Msys2 packages when possible. And Msys2 repository
provides version 0.3.9, released on March 2, which contains our patches.
No need for them here anymore, no need to make custom builds.
This is a new feature I implemented in the crossroad cross-compilation
tool. Msys2 repository has more packages and they are more up-to-date
compared to Fedora and Suse cross-built packages (the 2 other available
sources for pre-built Windows packages).
This allows to simplify a lot the dependency preparation for the Windows
CI, and speed things up.
json-c has 2 build systems (autotools and CMake) and it seems their
autotools broke with recent changes. I will report upstream. For the
time being, we may as well switch to CMake build.
Previous OpenBlas patch fixed the crash with Sophos (see reports #3633
and #4246) but created a huge slowdown of startup because of a timeout
change and most likely OpenBLAS being loaded at startup during the
various verifications.
A new patch has been merged upstream to lower this timeout to something
more reasonable. Reporters confirmed GIMP now runs fine (neither crashes
nor very long startups).
See: https://github.com/xianyi/OpenBLAS/pull/2339
Actual patch contributor wants confidentiality to avoid leaking
proprietary information or whatever (I am not sure either what to be
scared of as it's all good and harmless to me, but let's respect the
request). See also #4246 for more details.
A few commands need to be performed the first time for glib to work
properly, and gdk-pixbuf loaders to be found. I add them in a wrapper
script so that it's easy to ask people to test the dev builds (even
though it's not necessary to run these commands each time, but who
cares!).
Let's make sure they are not pulled in as dependency of other packages.
This fixes the Win32 CI build now that we fixed the Pango minimum
requirement in meson files.
Rather than having the whole Win32 cross-build into the 'gimp' stage,
break the dependencies and GIMP-only builds in 2 stages.
Since apparently we need to keep the same structure for the native and
cross build (otherwise we don't get parallel builds; in other words, I
didn't find the possibility to set separate pipelines up), I move babl
and GEGL into the same 'dependencies' stage.
Finally I remove the -base rules extended into actual jobs, except for
`.gimp-base` (this is the only which makes sense as it is actually
common to the meson and autotools build).
It looks like Arch does not have mingw64 cross-compilers in core package
repository. It does have some package in the user repository (AUR), but
I assume that such a repository cannot be deemed as safe.
Anyway I still tried, but apparently these AUR packages have to be built
and when I tried, I got this error:
> ERROR: Running makepkg as root is not allowed as it can cause
> permanent, catastrophic damage to your system.
Anyway it's all a big mess. Then I tried to move the cross-CI to Debian
testing, which is anyway our base compatibility system. Unfortunately I
encountered like what looked like some glibc++ macro problem on some
packages (most likely because the pre-built packages I use are Fedora
ones which likely uses a cross-compiler differently built from the
Debian one).
So in the end, for simplicity, I use a Fedora image, then I am sure to
get a good match between the system cross-compiler and the pre-built
dependencies.
I did not commit on purpose because this is actually to be found in the
official flathub repository for GIMP (the concept being to keep the
stable and nightly manifests as sync-ed as possible) and I thought there
is no need to duplicate data. But since this is apparently confusing
people (cf. !91), let's just push it.
Note that even though this patch is taken from the org.octave.Octave
flatpak package, it is actually slightly different (I tweaked it because
we needed to build even less options from SuiteSpace than they do
apparently).
Several of our own translations of the Windows installer are unused
because Inno Setup corresponding translations are marked "unofficial".
This mostly means that the language files for these are probably old and
unmaintained, hence outdated. And these files are not bundled together
with Inno Setup release (though still hosted in their repo).
Nevertheless it doesn't make sense that we would just waste the work of
our translators here. Maybe Inno Setup localization is not complete, so
what? At best it could even encourage translators to contribute upstream
to Inno Setup. Let's just enable all our current localizations of the
installer and see how it goes!
Anyway Python deps does not work yet on the master build. That's
unneeded to build nightlies with Python support. Let's drop it for now
and add it back later when we will make it work.
This breaks the flatpak tests at the end when we prepared a future
<release> appdata tag. For released version, we must not do this, but
for nightlies, it doesn't matter.
This will set the default color profile directory under its path inside
the sandbox and therefore fix issue #15 of our flathub repo.
While I am at it, I do some cleaning and remove --enable-vector-icons
since this is now the default.
This is in particular necessary because permission in manifest
--filesystem=/usr/share/color/icc does not work as expected to mount the
system repository inside the sandbox at the same path.
See commit 8f3bf7b175d44118f9146e31191fcb7558614b31 from flathub
repository for org.gimp.GIMP.
Let's go with a simpler manifest since soon it will be built on a better
machine (our CI server) and to make scripts simpler.
Note that I also removed the old webkitgtk dependency, but actually I
realize the GNOME runtime does not have the new dependency as well,
since the runtime actually has a webkit2gtk-4.0 instead. We'll have to
look into this.
This flatpak is tested and working.
I have not yet verified though if it has the maximum options enabled
(guessing not), but only that we had at least the required dependencies
at least.
When we will merge, it will become the nightly flatpak and the current
nightly can be used to follow the 2.10 (future) branch.
This is the script I have been personally using for some time now to
build the devel and nightly flatpak on my server regularly through cron
files.
It is not the best script, arguably, and we can most likely do better.
But for now, it worked for me and is a first step toward maybe making
official nightly flatpak builds soon.
In particular a bunch of dependencies were moved to the BaseApp.
Dev flatpak is actually a bit useless now (since last dev release is
older than stable). But I keep the dev manifest around to make it easier
to update for the future when we'll have a new dev release.
Fix the registry path where uninstaller information is searched for
during installation, so that old GIMP versions are properly
uninstalled before installing a new version.
This fix has already been included in the 2.10.0 installer.
In order to bundle the MyPaint brushes, GIMP needs to be configured
with --enable-bundled-mypaint-brushes, and the brushes need to be
present at $GIMP_DIR32\share\mypaint-data.
The Win32 build was not taking the new path of git-version.h into
account and was outputting these errors:
> ../build/windows/gimp.rc:2:10: fatal error: git-version.h: No such file or directory
> #include "git-version.h"
> ^~~~~~~~~~~~~~~
I completely forgot to rename the appstream file according to the new
ID. While doing so, I also make it a .in.in file, with initial
processing by the autotools. Indeed I need @GIMP_COMMAND@ to be replaced
by AC_CONFIG_FILES().
Finally I fix a badly closed XML tag (which reminds me I should always
test a commit, even when it's a simple non-C 1-liner change!).
...the installer component
The warning is actually not only about the installer, but also covers
the installed version of GIMP. The warning was resurrected from commit
6f0bb88e43 and slightly reworded.
In configure.ac, add --enable-windows-installer option (off by
default), which should be set to generate the necessary files for
the installer translations during the build. Using this option is
only supported when building from git, since the installer files
are not included in source tarballs.
Rename setup.isl.desktop.in to setup.isl.in, and instruct intltool
to treat it as an .ini file explicitly.
Delete generated message files during make clean.
Remove the installer graphics credits, since we use a different set
of graphics in master, and will probably use yet different set of
graphics for 2.10.
Add strings for the MyPaint brushes component. The setup script in
git doesn't use those yet, but this lets translators start
translating them.
Use intltool for managing the translations for the Windows
installer, instead of manually maintaining the translated message
files.
The message files are generated in the source directory, under
build/windows/installer/lang, as part of the build, and can be
subsequently used to build the installer, as before.
Though the bug was mostly fixed, it seems to still happen on Windows XP,
where apparently no content type had been registered for SVG.
GTK+ developers don't seem too keen to patch GTK+ 2.24 for a platform
which they don't support anymore.
Also if not mistaken, GIMP does not officially support Windows XP
anymore either. A patch though has already been provided by Edward E.
and it really doesn't look like it could break anything since it just
adds a last "else if" when everything else failed (and inside a #ifdef
affecting only WIN32 builds).
So let's just add it in our builds at least. We still don't have support
for it, but I see no reason to just refuse a minor patch which won't
break anything else.
The BaseApp json used for all 3 builds (stable, dev and nightly) is
common so just keep the one made available in the flathub upstream
repository. The patch also applies to the BaseApp only.
... interfere with GIMP UI events
Add a GTK+ patch to ignore top-level transparent windows when
looking for the top-level GDK window at a certain pointer location,
in the Win32 GDK backend.
The BaseApp manifest got a bunch of fixes, mostly to have all the
dependencies successfully build for aarch64. The current manifest should
therefore correctly build GIMP for i386, x86-64, arm and aarch64.
See: https://github.com/flathub/flathub/pull/124
The existing graphics are still from 2.8 (specifically, the have a
"2.8" caption, so we can't use them for 2.9); these are the
graphics used for the 2.9.6 installer.
... makes the list scroll down and select the next item
Adjust the file association list height to be a multiple of the
item height, to avoid this issue.
I had an error in the previous commit (2 args in 1). Also adding access
so that the file `bookmarks` is visible from the contained GIMP
(otherwise bookmarked folders are lost in flatpak and that's bad
experience).
For some reason, the "cleanup" tag won't delete the files produced by
the BaseApp inside the main GIMP manifest (I still need them for build,
but want to delete them for runtime).
This is a workaround to delete them with a few command lines for now.
See: https://github.com/flatpak/flatpak/issues/1082
Keeping all dependencies inside the main manifest is very annoying
because flatpak-builder will check them every time the package is
rebuilt. Worse, sometimes the cache won't be hit (even though it should
have), resulting into a rebuild of many dependencies. I create a BaseApp
build which is the recommended process (and not creating our own runtime
based on GNOME one's, as I first thought) which won't need to be built
as often as the main manifests. The other advantage is obviously that
this BaseApp can be shared between the dev and nightly (and likely even
the stable later) builds. I will only keep differences inside the main
manifests (for instance lcms2 which requires a higher version on master
than on the GNOME runtime and the last dev release).
I also move webkitgtk as the first dependencies since it takes too long
and flatpak uses a sequential dependency graph (so any change to a
previously listed dependency, even when actually unrelated, was
triggering a rebuild of webkitgtk!).
Only remaining issue is that I don't manage yet to run the cleanup of
the BaseApp at the end of the main manifests (for files needed for
building, but not at runtime).
So I discover today that there is an option --ccache to request
flatpak-builder to compile using ccache, which is obviously a great idea
when rebuilding the same deps too often. Update the howto with the info.
Not sure why, because flatpak-builder used to compile webkitgtk just
fine, but not anymore. I get now a "call of overloaded ‘abs(gdouble)’ is
ambiguous" error. It looks like some header may have been updated in the
flatpak environment and causing multiple abs definitions. Using fabs()
instead works around the issue.
This is a first try at creating a docker-based build environment for GIMP.
The file has grown alongside building babl, GEGL, libmypaint and GIMP on a RHEL host running Docker. The main purpose of this image will be to serve as a base for building packages like Flatpak, AppImage and others, and serving them from download.gimp.org
it is not optimized at all, pulls in some extra packages, misses some optional dependencies, builds as root, ...
build-export is actually a low-level tool used by flatpak-builder. When
using it directly, debug and locale extensions were not extracted as
separate extensions (unless tweaking complicated command lines), ending
up with a huge GIMP flatpak with the current procedure.
Since flatpak 0.9.5, the option --export-only has been added to
`flatpak-builder` so that the build and the export can be made in 2
separate steps while using the high level procedure.
See: https://github.com/flatpak/flatpak/issues/824
In particular, I didn't have the correct metadata because I was missing
appstream-compose. This should be a dependency of flatpak. But for the
time being, let's just have a note in our howto.
The GNOME sdk runtime used to be built which flags not appreciated by
older CPUs. This has now been fixed. Let's get rid of our workaround.
See https://github.com/flatpak/flatpak/issues/143
See the upstream issue where python2 would crash on some platforms, yet
not others: https://github.com/flatpak/flatpak/issues/143
For this, I disable introspection on GEGL (which is anyway not useful
in the Flatpak build, I believe), and wrap gdbus-codegen in a temporary
build script for GIMP to force it to use Python 3 (I can't force
globally the $PYTHON environment variable like on Webkit because GIMP
needs Python 2 as dependency).
For some reason, Webkit would not build with Python 2 on a different
machine though it did on mine. Let's just force Python 3 usage (for
build only, not as a runtime dependency) by setting the $PYTHON
environment variable during this build. Anyway Python 3 is available
from the base build so it's not a new dependency.
We can actually group modules together to mark their relationship,
and easily deactivate them together if needed.
I grouped all Python modules, Exiv2 under GEXiv2, ilmbase under
openEXR, and finally libpng and lcms2 under a "duplicate-dependencies"
module to indicate that these modules are available in the runtime
(under a lower version, below our requirements) so we should check
whether to remove them, upon runtime update.
The build has been fixed by a patch from Upstream:
https://bugs.webkit.org/show_bug.cgi?id=156510
Note that WebKit takes hours to build, so if you just want to test
Flatpak, you may prefer disable this module again.
We now have a full-feature Flatpak build, except for libgudev, which
means that special input devices probably won't work in the GIMP
Flatpak. It is to be noted that Flatpak developers told me that
libudev won't work inside the sandbox anyway. A portal will have
to be implemented. See:
https://github.com/flatpak/flatpak/issues/12#issuecomment-276016074
The last stable WebkitGTK is 2.14.3 but that apparently corresponds to
WebkitGTK4 (which is already in the GNOME runtime). We need an older
version, and the last available would be 2.4.11.
I disable it though because the build failed in my test but let's save
my work-in-progress.
This is necessary for opening files from the network.
It is actually possible to allow only some named service bus which
would be much more in line with proper sandboxing. But I can't find
exactly what is the service name used for gvfs (which is the virtual
filesystem apparently used internally for remote file access in GIO).
Also I'm pretty sure we must use other buses for various services. We
should make a list of what service bus are necessary for normal usage.
For the time being though, let's just have a flatpak build with as many
features as possible.
Based on Alexander Larsson's original nightly GIMP flatpak. I updated
most dependencies, changed some options, and added some dependencies
whose versions were now too low in the runtime (libpng and lcms2).
I also use the new "--filesystem=xdg-config/GIMP" option, made upon my
request, which means we will want to use a recent flatpak.
Many options are still missing but that's a start with a working
flatpak manifest.
Jernej Simončič