Copied over from the original separate work in:
https://gitlab.gnome.org/Wormnest/gimp-file-plugin-tests
After that further improved and changed and added more file format
tests.
Added meson.build files to integrate it in our build, but we do not
install it for releases.
This regression was added by c064148a and it's similar to fb7a9954. Now, that
we are using 'interruptible: false' in custom rules, this is probably fixed.
Also, organize a bit better some pipelines with official GitLab .yml wizardy.
I was hesitating to use "<<: *" since this creates a new layer of complexity,
but this was the better way and the trick have a distinct marking by the way.
For our two "development" sub-stages, the expiration is as follows:
The 'dependencies' stage artifacts continues to expire in 2 hours.
Now, all 'gimp' ones expires in 2 days (to avoid time zone limbo).
For "stage"/tests and "production"/dist, the expire time depends
on the source of the CI pipeline. If the artifact is oriented to a
'short-span' source (e.g. MRs and commits), it expires in 2 days.
If is oriented to a 'LONG-span' one (e.g. web, schedule), 8 days
(we choose 8 to have a +1 day artifact just in case if something
goes wrong in the weekly schedule day, which isn't rare actually).
Partially reverts af79bbe0 (regarding .debug "extraction" in cross builds)
Now, we call the split debug script from the main bundling script, which makes
similar to our macOS .app bundling script. This cleans a bit of code in .yml
and make things clearer to the mere mortals in the scripts.
Native arm64 and x64 TWAIN binaries are not built anymore. No need to
discard at dist time what doesn't exist. (32-bit TWAIN not affected)
Also, start to using our own ARM64 runners to do the dist. This is more
reliable since the shared x64 runners can (and indeed) cause long queue.
They were generating a distracting output in CLANG* shells, as noted by
@lillolollo in a comment from MR: Infrastructure/gimp-web-devel!65
In the process, make AppImage and Windows (native) scripts use these
variables, without hardcoding the same variables from .yml anymore.
Creating a temporary config directory for the in-build GIMP (run as a tool or
for unit-testing) is not done as a build target anymore, but in the
in-build-gimp.sh script as a unique temp directory, then cleaned out on exit.
This has a few advantages:
- It is properly cleaned out once the build ends (instead of leaving a full
config dir as trash inside the build dir).
- It is not reused from one build to another (with risk of carrying bugs and
issues over).
- Every use of the in-build GIMP will have its own config directory, and in
particular when they are called in parallel.
As a side update, make sure that all `gimp_exe` runs depend on
`gimp_exe_depends`.
The new dist job can produce two artifacts: a *.msixupload file
ready for submission and/or a *.msixbundle only for testing.
See the commit about the script for more details.
As hinted in d09a2a6f
We now use the word 'bundle' to signify "program files in the same prefix"
(e.g. .appimage, .zip, .app). This is in line with our source and dev-docs
(just take a search in the repo). So, appimage and windows scripts changed.
The word 'package' normally means "program files distributed for install in
the same prefix or not" (e.g. .deb, .msi, .dmg). This is in line with CMake
naming of some commands, but meson prefers to call 'dist', which we use more.
So, this partly reverts some things of GNOME/gimp!1171 and reinforce others
for even more "rationality" in the overall build structure of GIMP.
This is a well know problem documented in GitLab docs that happens when
the CI isn't properly configured that was causing double pipelines in MRs.
Let's fix it, and reduce some redundancy.
I suppose the intention of having this stage at the end was to make clear: no
job depends on it. However, this place is not common or ideal but one of two:
1) before building, which is more linear; or 2) after building (the choosen)
even if this isn't dynamic analysis (tests), since they are both related and
some compilers and build systems do static analysis. Anyway, not after dist.
The new place is a bit more clear to devs used at "build, test, deploy" praxis
and maybe will be now better noticed by the existing devs before distributing.
It's not perfect, since I explained above that there are two places/approaches.
Obviously, the stage continues to be OPTIONAL to pass, nothing was changed.
---
Also, dropped the deprecated "only:" (in clang-format job) in favor of "rules:"
and made the artifacts paths of clang-format and cppcheck more consistent.
AppImage is pretty fast to make, like the win crossbuild; and portable,
being very appropriate to do quick tests on Linux when pushing to git.
The overall organization of Debian jobs was changed to take advantage
of this and make things less complicated (but less clear at first sight).
I reinforce that this was the most efficent way to make the AppImage.
This fix doesn't feel right. The whole point of having a generated manifest from
the build was to use it for our flatpak job (through job dependency). Right now,
it has been rendered useless. We must either re-implement the proper job
dependency or find another way to fill the proper "command" with the right
version (maybe just a standalone script to be used in Gitlab CI and which we
could tell about to people who wish to build the nightly flatpak themselves).
Anyway for now, it should do as I don't want to spend too long on this. It
should fix this error in the log generated as artefact of the "gimp-flatpak-x64"
job:
> ../build/flatpak/meson.build:6:2: ERROR: File org.gimp.GIMP-nightly.json.in does not exist.
This stage was introduced in 076e4d68 and it fulfilled well its purpose. But
the side effects were: more (sometimes slow) uploading and downloading of
packages and artifacts. Even if we have caching, this is far from ideal.
Now, the so-called "packaging" jobs were moved inside in the "build" jobs.
The advantages are: just one pacman install (ideally cached from deps job);
direct packaging without waiting for upload/download; and less confusion,
since "packaging" naming was implying that the .zips are somewhat official,
which isn't totally true, and sometimes overlaps the dist/deploy concept.
---
Also, the flatpak job now uses config.h instead of triggering a GIMP build.
In other words, I'm reverting part of my work in some old commits.
Now, the CI will auto cancel jobs after a new commit is pushed and retry when
the runners fails. The first case is frequent, the last is rare but happened.
Since !1171 the 'before_script' was being used by me for organization, but
this is at least unexpected to "mainstream" .yml pratice and can be wrong.
For example, the universal variables can't be properly used this way.
So, let's use 'before_script' only for the (official) purpose: save code.
This isn't directly related with #10922 and does NOT affect 32-bit support.
After talk with @Jehan, author of 'crossroad' and main dev of crossbuilds,
we decided that the x86 crossroad builds can be dropped by these reasons:
1) 32-bit crossbuilds weren't being built. They are used since c8ec0ae7 but
after the native 32-bit builds they ended in a state that they are the only
pipeline that isn't automatically triggered in any way, a complete limbo.
2) 32-bit crossbuilds weren't being autonomous. They are not working fine
with wine32 so the win-x64-cross artifact is a 'need' and now debian-x64,
so we were spending way more time fixing than using the 32-bit cross jobs
(sometimes the 32-bit cross jobs are just broken and we even noticed).
We have other jobs not autonomous but they don't have additional bugs
like not being able to build with .mng and .wmf (32-bit cross only bugs).
---
The x64 crossbuilds will continue to be supported. They are very useful to
develop on Linux, and test MRs on Windows. I'm an assiduous user of them :)
This was added in a22393a6 and e96b751e, following old 2_build-gimp-msys2.sh.
But, according to my tests (local and in the CI, in both Debian and Windows),
meson creates this in setup time so the manual folder creation is redundant.
* Move common things to the new .default section extensible for all jobs
* Reorganize order of job elements to be way more consistent and rational:
from general (pipeline/rules) to specific (job artifacts) in a liner way
* The order of rules was changed to be a bit more easier to understand
- Drops 'dependencies' which is redudant, and 'needs' have more options
- Drop apt-cache which lost use and it is sometimes troublesome to cache
+ First try to boost the use of ccache in all build jobs, when possible
+ Fix the problem of non-failing jobs introduced by GNOME/gimp!1171
This commit uses definitions and references to reduce repetition of CI code.
Some people (like the GitLab devs?) prefer (almost imposes) this to be done in
multiple .yml's using 'worfklow:' but this makes the CI way more complicated
and very slow to read. So, let's just keep simple in one human readable file.
---
Other changes are:
- The custom native arch pipelines now they always generate a .zip package.
This helps to ensure that everything is in place and facilitate testing;
- The pacman update (-Syyuu) was removed since it is auto done by runners;
- The ccache config was moved to .yml. This does NOT affect local builds
since meson saves ccache by default for user in '%LocalAppData%/ccache'.
These commits introduced sensible drawbacks in our CI file organization.
Some of them I already commented in the MR but was "ignored" back then
probably due to the complexity of the MR. So, the let's revert:
- Unecessary repetition of libomp installation (4 times, now 1)
- Unecessary repetition of submodule initialization (6 times, now 4)
- Creation of new variables in the hope of more clarity. But,
they aren't consistent with the gimp-web-devel instructions
nor with other systems variables. This is terrible to undestand.
Please, let's not reinvent the wheel since this is a bad example to
packagers, since they tends to isolation and overly-engineering.
Also, made some little adjustments that I should have done in previous MR.
---
This MR was splitted. Even more simplification will be done in the future.
Since dist-installer-weekly doesn't depend on gimp-debian-x64 anymore, we need
to grab the config.h from one of the other dependencies. Let's use the one from
gimp-win-a64 (though it could have been any other Windows build job).
Fixes:
> Get-Content : Cannot bind argument to parameter 'Path' because it is an empty string.
> At C:\_r\_builds\vJWzEqDv\0\GNOME\gimp\build\windows\gitlab-ci\4_dist-gimp-inno.ps1:13 char:39
> + $GIMP_VERSION = Get-Content -Path "$CONFIG_PATH" | Select ...
> + ~~~~~~~~~~~~~~
> + CategoryInfo : InvalidData: (:) [Get-Content], ParameterBindingValidationException
> + FullyQualifiedErrorId : ParameterArgumentValidationErrorEmptyStringNotAllowed,Microsoft.PowerShell.Commands.GetC
> ontentCommand
The dependency to libomp apparently came with the move to CLang.
Fixes:
> /builds/GNOME/gimp/_install-debian-x64/bin/gimp-console-2.99: error while loading shared libraries: libomp.so.5: cannot open shared object file: No such file or directory
Now the development and stable logos will be generated from gimp-data.
In other changes, the gi-docgen logo is installed as a symlink using
install_symlink() which exists since meson 0.61.0 so I bumped our meson
dependency (in practice we were already using this function anyway and Debian
bookworm has meson 1.0.1 so it's all good).
Finally I don't install a wilber.png anymore, which was only used by script-fu
testing, and which was the same as gimp-logo.png (except 256x256 instead of
128x128). Unless mistaken, all script-fu tests loading this image still work
with the change. The only one where I needed further change was buffer.scm
(which was checking the dimensions).
See gimp-data@9aa6e35.
- With last commit, the Windows installer pipeline doesn't depend on
"gimp-debian-x64" job anymore since a native Windows build is now able to run
GIMP (or gimp-console) as a build-time tool as well. It makes the Windows
installer pipeline (and full custom native builds) self-sufficient.
- On the other hand, "gimp-win-x64-cross" and "gimp-win-x86-cross" now require
"gimp-debian-x64" since cross-compiling GIMP now requires a native GIMP in
order to generate some image data (such as the splash image, and probably soon
logo or icons, etc.). See gimp-data@5a03c71.
- Getting rid of "image-win-x64-cross" and "image-win-x86-cross" in favor of
"image-debian-x64" for all Debian as well as the cross-compilation jobs. They
are all based on the same Debian image (it was debian:bookworm for native
Linux jobs and debian:testing for cross-builds; now it will be debian:bookwork
for all) and it's just a few more packages (cross-compilation C and C++
toolchains) for the cross-builds. Moreover now the cross-builds also need the
native GIMP binary around, therefore native dependencies are needed as well.
It makes sense to factorize all 3 images into 1.
- Make sure we don't build bindings when cross-compiling since these won't work
in this case.
Meson subprojects just have too many problems and limitations and I can foresee
the maintenance headache and the future incoming false-positive bug reports if
we start using meson subprojects.
Comparing to the simplicity of git submodule which also has much better
notifications to help people understand when the submodule is not in sync and
how to remedy to it.
See commit gimp-data@c364adb explaining the main reasons in detail.
In our efforts to use Clang, now the nightly (in fact weekly) flatpak
is built with Clang too, not only GIMP but also the dependencies.
* However, not aalib. We welcome fixes regarding this cursed lib.
Additionally, updated some build options from some deps.
This commit unites two scripts (.sh + .bat) in one .ps1. PS was choosed, since:
1) We don't need MSYS2 to distribute with Inno at first. Now, who wants to
just distribute GIMP can do it easier natively without a separate environ.
2) PS is actually pretty solid and have a good number of cmdlets, unlike the
ancient CMD, which barely supported more than 9 parameters in .bat files.
Consequently, some redundant variable checks have been removed from the .ISS.
Also, changes the name of the job and script for better consistency with the
upcoming MS Store job.
Bruno Lopes