This is not a fix, only a workaround for the CI to at least not fail
randomly, for the time being.
Basically since meson is running parallel jobs, even when the build
fails because git-version.h doesn't exist yet, while a depending file is
being built, it will still be built immediately after, despite the
failure. As a consequence, re-running `ninja` immediately after (i.e.
running `ninja || ninja`) will make the second build work. If it doesn't
then it's another issue which has to be fixed. But at least we work
around this known race condition in the meson build for the time being.
It's very ugly, but better than current situation. :-/
By enabling this option on Linux, not only will the installer language
file be generated, but the `make check` will also now validate that the
lang list is consistent with existing gettext files (see commit
8a42c6ccc2). So it's useful information to know this for instance as
soon as some translators add a new localization.
Also oppositely remove the option on the MSYS2 native Windows 32-bit
build. For Windows, we only need this option once, as we use the
language files generated by the 64-bit build.
The goal is to make it easier to understand which pipeline is run when,
for future maintenance, and also to make it easy to trigger a specific
pipeline through Gitlab pipeline interface.
- Adding a patch sent to me by Sylvie Alexandre meant to help aalib
build on MSYS2 for Windows 32 and 64-bit.
- Additionally, add an additional patch from myself because it was still
not building properly.
- Also update the config.guess|sub files because the original ones (from
2001!) were just too old and not properly recognizing the host mingw
system (especially the 64-bit one apparently) in the MSYS2 CI jobs of
GIMP.
- Finally regenerate the whole aclocal/libtoolize/autoconf/automake
build system because these old files just don't play nice with recent
autotools though the source files still regenerate fine (despite with
some warnings, but nothing blocking).
- Add crt-git dependency because libws2_32 is in there.
We had an autotools build stopping at `make check` and a separate one
for `make distcheck`. This just seems very redundant hence a waste of
resources.
So let's drop the job gimp-autotools-debian then add a `make check` step
to gimp-distcheck-debian.
Also as a side change, I move the cppcheck to being a scheduled job.
It's not such resource intensive job, nor did it take much time, yet
it's not like we use this information constantly. Moreover it never
fails anyway (so it's not like it gives much information on a per-commit
basis, unless we explicitly look into the resulting files) and with the
ability to run custom jobs whenever we want, this is far enough if
sometimes we need to generate the cppcheck analysis more frequently.
The rest of the time, having 2 jobs of this a week (our current
schedule) is far enough.
Run InnoSetup in the Windows CI to build the installer from both the 32
and 64-bit builds.
Current limitations:
- No installer signature yet.
- Dependencies will have to be checked more thoroughly.
- Apart from babl and GEGL, we may want to make custom builds of any
package which has a patch in build/windows/patches/ (Windows-specific
patches) and build/patches/ (all platform patches).
- Plug-in interpreters (Python, Lua…) don't work. This will need to be
looked at in detail.
Globally this first automated installer build works fine though, as I
could install it in a Windows 10 VM and GIMP ran fine! So it's a first
step towards fully automated releases for Windows.
Having them at each commit is counter-productive, first because these
builds take so long and second because there seems to be quite few
Windows runners. So we end up constantly waiting for CI jobs from
previous commits (so we are just constantly waiting).
This is resource over-usage. So instead, I'll just set this in scheduled
jobs. For release preparation though, we'll have to set up a workflow
later to trigger these jobs off-schedule/on-demand (I can see there is a
Gitlab interface to do this), but this can wait for when the installer
is fully generated by the CI anyway.
Note: Vala API doesn't build well on the 32-bit build. Not sure why (the
meson logs for GObject Introspection build are just as empty as ever),
but it won't generate the VAPI. So I disabled the option on 32-bit.
This new job resulted in a package which allows to run GIMP on Windows
(as tested in a VM; at least it starts, I can create a new canvas and
paint). Of course I think this will need to be tweaked a little bit
more, as I'm sure we miss things here and there.
At the very least, even though I add the Python and Luajit binaries,
GIMP on Windows didn't find them. This will need to be investigated.
Also it looks like opening from a remote location may not work. Not sure
if this about a missing GIO module or maybe something which works
differently on Windows (I was not even able to drag'n drop from the
browser!). Anyway this needs to be looked at as well.
Note that gdk-pixbuf-query-loaders is apparently unneeded when GIMP is
built this way (unlike with our crossroad build).
All this to say that this is still an early attempt to full CI build for
Windows.
It doesn't invalidate the crossroad build, because cross-compilation
builds from Linux will always stay very important for Linux developers
to be able to easily fix Windows bugs too; yet the crossroad build has 2
major issues:
1. We haven't figured out yet how to run GObject Introspection tools for
cross-builds, so the crossroad builds are not full-featured (and this
is quite a major feature we are missing!).
2. Also I will want to run the installer in the CI at some point and the
one we use can only run on Windows itself AFAIK. We could try to run
it through Wine, but still anyway the point 1. is already quite a
blocker, let's do the simple thing.
Note that we will likely want to move to meson for this build, because
autotools is very slow on Windows. But as long as the few blocker meson
bugs are not fixed, let's stick to the slow yet good build.
These are interesting and may find very specific bugs from time to time,
but the usefulness is rare enough not to warrant to run at each commits.
This is just a waste of resources.
For scheduling finesse (in case we want to separate these in separate
scheduling), also rely on the existence of variables during scheduling.
Finally make sure that the non-scheduled builds are not run in schedule
pipelines (they are already run far enough).
The build rules were highly inspired by other projects on GNOME's
Gitlab. All of them used to build with ccache. It worked fine for the
main build, but completely broke GObject Introspection build on both
babl and GEGL. And the worse thing is that meson was absolutely not
displaying the error, just saying it failed (even in verbose mode). A
lot of time wasted trying to debug.
Therefore let's get rid of ccache, but only for babl and GEGL. Keep it
for GIMP itself as it works fine there.
Other minor changes:
* Build from the build dir, rather than source. The other way around
works too, but I actually find commands simpler this way.
* Adding artifacts.
Just an initial test to get a hang of the thing, mostly inspired from
GTK gitlab-ci rules adapted to our build.
All in one job (deps, babl, GEGL, GIMP itself) for now, for simplicity
of debugging. We'll see later to break this into CI sub-jobs.
- Only publish the bz2 tarball because that's what we currently provide
on download.gimp.org (let's see in the future for the xz tarball).
- Generate also a sha512 checksum. It's better to do it on the CI rather
than on the download server, otherwise it wouldn't protect against
transfer errors (from gitlab to download server).
- Rename the checksum files to ${filename}.SHA256SUMS (512 respectively)
for easy download without name clash with the global files listing all
the previous releases.
- Disable all meson jobs for tagged releases. They are currently not
reliable and may fail randomly (see issue #6257), even though without
code problems (this is also one of the reasons why autotools is still
our official build system and meson is still deemed experimental).
It's ok for regular builds, but not for tagged builds, which we need
as reliable as possible. The really important job is the distcheck one
for tags.
This reverts commit 32434d9fc3.
Argh sorry. I was planning on making a branch for this (not directly on
master!) and also the patch should obviously have been applied on a GTK
tree, not GIMP's. I'm probably tired! Reverting.
We were not building it by not installing libmng until now. But graphviz
must have pulled libmng. So disable explicitly the option in GIMP
configure step.
This reverts commit e869a11270.
Mitch could reproduce the issue and made each .actions file into
respective CLEANFILES in commit 0052803313. At least now the distcheck
job works, so let's just make its failure forbidden again.
I really don't like to flag the distcheck job as allowed to fail, but
the issue we have with it right now (#5790) is very annoying and I have
no idea where the weird uncleaned files come from. I can't reproduce
this locally and these files are seemingly never created here during a
distcheck.
Since it makes all our pipelines fail, this makes it harder to diagnose
and find real other bugs, so let's allow failure until we figure this
out.
For Python, Lua and Javascript, make the option boolean (with 'yes'
being the default). No need of a warning when not installing the
plug-ins as this would have been disabled explicitly anyway. When
installing the plug-ins, only make interpreter checks as precautionnary
verifications which don't actually change anything (except outputting
some warnings if interpreters are not found). Basically for these 3
bindings, the interpreters are only runtime dependencies anyway. So it
doesn't matter if they are not available at build time. In particular,
we get rid of the 'force' option.
Vala rules do not change as the vala compiler is indeed needed at build
time and current checks work correctly. I just add a "Vala plug-ins"
line in the summary message of the meson configuration, as it was
missing.
This should give a nice name to distribution archives so that they are
not all called `artifacts.zip`. Names will better describe their
contents (target OS or source and short commit hash, because for CI
builds, it's important to know which commit is being tested).
Also replace CI_COMMIT_REF_NAME in other artifact names by
CI_COMMIT_REF_SLUG. Otherwise if a branch has a slash (quite common in
branch names), only the part after the last slash is used for archive
naming.
Finally immediately exits from dependency build with error code (!= 0)
if `crossroad install` command failed.
… the CI.
There are 2 finale steps before finale binary distribution on Windows.
We must compile the GSettings XML schema files and register GdkPixbuf
loaders (for file format support in the GUI).
I used to provide a wrapper to be run inside Windows before first GIMP
run. Never did I realize that I can compile the distributed GSettings
schemas with the native `glib-compile-schemas` (works fine in my tests).
As for the GdkPixbuf loaders, we inspect DLL libraries, hence we do
require the target `gdk-pixbuf-query-loaders` which is unfortunately a
Windows executable. Yet it seems to work fine with Wine, so let's be
done with it in the CI instead of requiring manual steps from testers of
the CI builds. Then a few `sed` calls are enough to make the path in the
produced text file relative instead of absolute (which works fine, again
in my tests at least).
This means that I don't have to distribute the 2 binaries and the DLLs
they depend on anymore. Moreover let's remove the wrapper (but still
generate one which just calls GIMP so that we call it from the tree
root, where it's much less messy).
Note: I failed to install wine32 (32-bit Wine) on the Gitlab runner.
After following all instructions, I encountered weird errors. So
instead, I just make the win32-nightly job depend on win64-nightly and
copy `loaders.cache` from one to another, as it is a
platform-independent text file (as long as we provide the same GdkPixbuf
loaders on both of course, which we do).
The main purpose of these jobs is to only package the strict necessary
for a working GIMP under Windows, i.e. getting rid of all unnecessary
executables, and inspecting binary dependencies recursively to only
package used DLLs.
The dll_link.py script is taken from Siril codebase (see commit a86e82a8
on Siril repository, by FlorianBen). This was a very nice idea, and
makes for much smaller test archive (Siril is also GPLv3 so licensing is
ok for the reuse, also anyway it's just a small independent build
script).
Moreover having it as a separate job allows to have artifacts with only
the finale distribution (artifacts on the build job also have the build
directory and the whole prefix, which we want to keep in order to debug
when needed).
Hopefully I am not missing anything. Siril seems to package more, like
various gdk-pixbuf-*.exe, gspawn-*.exe and gdbus.exe. I am wondering if
these are actually necessary. I could run GIMP fine without these in
quick tests, but I guess I'll have to investigate a bit more to figure
this out. That's what nightly builds are for, after all, so hopefully
people will report if we miss some runtime dependencies.
I don't add this at the end of the distcheck job to make the interface
clearer, and also because the distcheck job will have full build
artifacts (allowing to debug a failing distcheck if necessary), whereas
the `sources` job will just publish tarballs and SHA256 sums generated
from these tarballs. Simple, clean.
The Linux CI job names are too long and are not recognizable on the web
GUI unless you hover the widgets with the mouse to read tooltips. Remove
the "/testing" part (if people want to know exactly which Debian we use
for our builds, they can always look at the script) and move left the
differenciating parts (i.e. autotools/meson/clang/distcheck) so that
these are visible in a glance, even when ellipsing long job names.
Problem with the CI is that the source is straight in our base directory
and therefore installed dependency files as well as cache or built files
are in subfolders.
Let's try to ignore as much as I can see. This should avoid a bunch of
warning and errors during report generation.
Seems that msys2 packages can sometimes be compressed as `.tar.zst`
instead of `.tar.xz`. I updated crossroad to handle this case. It now
requires the additional pypi `zstandard` package.
This is a new feature I implemented in the crossroad cross-compilation
tool. Msys2 repository has more packages and they are more up-to-date
compared to Fedora and Suse cross-built packages (the 2 other available
sources for pre-built Windows packages).
This allows to simplify a lot the dependency preparation for the Windows
CI, and speed things up.
`gegl` binary is being run for icon generations on gimp-2-10.
(cherry picked from commit 97549081fd)
Note: this commit is so far not needed on master as we don't call gegl
during build time unlike in gimp-2-10. But we could at some point.
Better to be thorough.
Jehan