use `download-ci-llvm=true` in the default compiler config
1ca2708e77 made it so that the `src/llvm-project` submodule has to be checkout for `download-ci-llvm = "if-unchanged"` to know if the submodule has been changed, but that is not required, if the submodule hasn't been checkout it cannot have been modified.
~~This PR restore the previous behavior by only updating the submodule if it has already been checkout.~~
This PR makes `download-ci-llvm = true` check if CI llvm is available and make it the default for the compiler profile, as to prevent unnecessarily checking out `src/llvm-project` with `"if-unchanged"`.
r? `````@onur-ozkan`````
enable -Zrandomize-layout in debug CI builds
This builds rustc/libs/tools with `-Zrandomize-layout` on *-debug CI runners.
Only a handful of tests and asserts break with that enabled, which is promising. One test was fixable, the rest is dealt with by disabling them through new cargo features or compiletest directives.
The config.toml flag `rust.randomize-layout` defaults to false, so it has to be explicitly enabled for now.
Additionally teach compiletest to ignore tests that rely on deterministic layout.
Tests themselves aren't built with randomization but they can still observe
slight changes in std or rustc
Set LLVM_ENABLE_ZSTD alongside LLVM_ENABLE_ZLIB so that --compress-debug-sections=zstd is an option.
Use static linking to avoid a new runtime dependency. Add an llvm.libzstd bootstrap option for LLVM
with zstd. Set it off by default except for the dist builder. Handle llvm-config --system-libs output
that contains static libraries.
We were running testing on API 18, which was already out of support for
NDK 25, and some of the ancient behavior in that image was causing
trouble when developing `rustc` features (#120326).
Update to the current LTS NDK 26, and to its minimum supported API 21.
Fixes: #120567
This commit updates the support for the `wasm-component-ld` tool
from #126967 to conditionally build it rather than unconditionally
building it when LLD is enabled. This support is disabled by default and
can be enabled by one of two means:
* the `extended` field in `config.toml` which dist builders use to build
a complete set of tools for each host platform.
* a `"wasm-component-ld"` entry in the `tools` section of `config.toml`.
Neither of these are enabled by default meaning that most local builds
will likely not have this new tool built. Dist builders should still,
however, build the tool.
handle ci-rustc incompatible options during config parse
This PR ensures that `config.toml` does not use CI rustc incompatible options when CI rustc is enabled (just like [ci-llvm checks](e2cf31a614/src/bootstrap/src/core/config/config.rs (L1809-L1836))). Some options can change compiler's behavior in certain scenarios. If we don't check these incompatible options, CI runners using CI rustc might ignore options we have explicitly set. This could be dangerous as we might think a rustc test passed with option T but in fact it wasn't tested with option T.
Later in https://github.com/rust-lang/rust/pull/122709, I will disable CI rustc if any of those options were used (similar to [this approach](dd2c24aafd/src/ci/run.sh (L165-L169))). If CI runners fail because of these checks, it means the logic in run.sh isn't covering the incompatible options correctly (since any incompatible option should turn off CI rustc).
The list may not be complete, but should be a good first step as it's better than nothing!
Blocker for https://github.com/rust-lang/rust/pull/122709
use key-value format in stage0 file
Currently, we are working on the python removal task on bootstrap. Which means we have to extract some data from the stage0 file using shell scripts. However, parsing values from the stage0.json file is painful because shell scripts don't have a built-in way to parse json files.
This change simplifies the stage0 file format to key-value pairs, which makes it easily readable from any environment.
See the zulip thread for more details: https://rust-lang.zulipchat.com/#narrow/stream/326414-t-infra.2Fbootstrap/topic/Using.20different.20format.20in.20the.20stage0.20file
do not allow using local llvm while using rustc from ci
From: https://github.com/rust-lang/rust/issues/123586#issuecomment-2043296578
> Even if `llvm.download-ci-llvm` is set to true, `stage > 0` rustc will always use the prebuilt LLVM library which comes with ci-rustc. So I tried to use locally-built LLVM libraries in the ci-rustc by replacing the existing LLVM libraries with the locally built ones, and it appears that this is indeed a limitation of using `rust.download-rustc=true` as it fails with the following error:
>
> ```
> $ ./build/host/ci-rustc/bin/rustc --version
> ./build/host/ci-rustc/bin/rustc: symbol lookup error: /home/nimda/devspace/.other/rustc-builds/build/x86_64-unknown-linux-gnu/ci-rustc/bin/../lib/librustc_driver-a03ea465d8e03db1.so: undefined symbol: LLVMInitializeARMTargetInfo, version LLVM_18.1
> ```
>
> So, if `rust.download-rustc` is set to true and `llvm.download-ci-llvm` is false, I believe bootstrap should terminate the process (as it always uses prebuilt LLVM libraries from ci-rustc, there is no point to build LLVM locally) while parsing the configuration.
Resolves#123586
r? Mark-Simulacrum
This change makes `build.bootstrap-cache-path` option to be configurable with
`./configure` script, so it can be used like `./configure --bootstrap-cache-path=demo`.
Signed-off-by: onur-ozkan <work@onurozkan.dev>
This commit adds a `runner` field configuration to `config.toml` for
specifying a wrapper executable when executing binaries for a target.
This is pulled out of #122036 where a WebAssembly runtime is used, for
example, to execute tests for `wasm32-wasip1`.
The name "runner" here is chosen to match Cargo's `CARGO_*_RUNNER`
configuration, and to make things a bit more consistent this
additionally renames compiletest's `--runtool` argument to `--runner`.
Setting the bootstrap cache path to an external location can help to
speed up builds in cases where the build directory is not kept between
builds, e.g. in CI or other automated build systems.
This commit adds a new target called `wasm32-wasip1` to rustc.
This new target is explained in these two MCPs:
* https://github.com/rust-lang/compiler-team/issues/607
* https://github.com/rust-lang/compiler-team/issues/695
In short, the previous `wasm32-wasi` target is going to be renamed to
`wasm32-wasip1` to better live alongside the [new
`wasm32-wasip2` target](https://github.com/rust-lang/rust/pull/119616).
This new target is added alongside the `wasm32-wasi` target and has the
exact same definition as the previous target. This PR is effectively a
rename of `wasm32-wasi` to `wasm32-wasip1`. Note, however, that
as explained in rust-lang/compiler-team#695 the previous `wasm32-wasi`
target is not being removed at this time. This change will reach stable
Rust before even a warning about the rename will be printed. At this
time this change is just the start where a new target is introduced and
users can start migrating if they support only Nightly for example.
This allows building the compiler itself with one backend while using
another backend at runtime. For example this allows compiling rustc to
wasm using LLVM, while using Cranelift at runtime to produce actual
code. Cranelift can't compile to wasm, but is perfectly capable of
running on wasm. LLVM can compile to wasm, but can't run on wasm. [^1]
[^1]: The prototype of this still requires a couple of other patches.
in particular, this makes the `c` feature for compiler-builtins an explicit opt-in, rather than silently detected by whether `llvm-project` is checked out on disk.
exposing this is necessary because the `cc` crate doesn't support cross-compiling to MSVC, and we want people to be able to run `x check --target foo` regardless of whether they have a c toolchain available.
this also uses the new option in CI, where we *do* want to optimize compiler_builtins.
the new option is off by default for the `dev` channel and on otherwise.
This field was not functioning as described in its comment in `config.example.toml`.
Also, updated the default value to `true` to keep the bootstrapping behavior as it was before.
Signed-off-by: onur-ozkan <work@onurozkan.dev>
This way, we only update CONFIG_CHANGE_HISTORY for major changes, which is
much simpler (and updating example.toml doesn't make much sense)
Signed-off-by: onur-ozkan <work@onurozkan.dev>
Add two options when building rust: strip and stack protector.
If set `strip = true`, symbols will be stripped using `-Cstrip=symbols`.
Also can set `stack-protector` and stack protectors will be used.
In the future Windows will enable Control-flow Enforcement Technology
(CET aka Shadow Stacks). To protect the path where the context is
updated during exception handling, the binary is required to enumerate
valid unwind entrypoints in a dedicated section which is validated when
the context is being set during exception handling.
The required support for EHCONT has already been merged into LLVM,
long ago. This change adds the Rust codegen option to enable it.
Reference:
* https://reviews.llvm.org/D40223
This also adds a new `ehcont-guard` option to the bootstrap config which
enables EHCont Guard when building std.
onur-ozkan