The wheel version string has been broken for python>=3.8. It seems
recent version of setuptools_scm has changed something, so
downgrade the version for now.
Also this commit removes python 3.7 supports since it caused some version conflicts.
Tested: https://github.com/llvm/circt/actions/runs/6272413564
We were having problems with the universal MacOS wheels, so we
reverted back to just building the x86 wheels. But this still fails,
because these options are telling cibuildwheel to build for the
universal architecture on MacOS, which is not enabled.
Also install the OM linker as part of the minimal firtool installation.
This will be included in both the nightly and published releases.
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@sifive.com>
Change the GitHub Action that checks that the "main" branch isn't broken
for Windows builds to use the Unified Build/Test/Install Action.
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@sifive.com>
Change the workflow_call "package_name_prefix" of the unified
build/test/install GitHub workflow to be optional. This is only necessary
if an "install" argument is specified (and that is already optional!).
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@sifive.com>
Fix a bug in the UBTI workflow where this would error if no install target
was given. This workflow should work for any combination of optional
installation and testing.
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@sifive.com>
Fix a problem where the runTests input only worked correctly for
workflow_dispatch, but not for workflow_call.
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@sifive.com>
Add all available clang versions to the Unified Build/Test/Install GitHub
Workflow. This is done to allow for testing of any version of clang
that is available in the ubuntu-20.04 and ubuntu-22.04 runners. Not all
versions are compatible with all runners.
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@sifive.com>
Switch from using "clang-12"/"clang++-12" to just "clang"/"clang++". This
doesn't appear to be necessary and will help unify the future use of
GitHub caching.
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@sifive.com>
Enable sccache-based caching for the unified build/test/install GitHub
Workflow. Use sccache as opposed to ccache because the former supports
Windows (as suggested by the hendrikmuhs/ccache-action documentation).
Disable caching for Windows non-release builds as there is a bug involving
cmake and sccache.
This commit is the last piece necessary to begin replacing existing
build/test GitHub workflows used in CIRCT.
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@sifive.com>
The name of the artifact that include the nightly and published `firtool`
was changed from `firrtl-bin` to `firtool-bin`. This caused a bunch of
other CI to stop getting the latest version (and will break CI on the
1.52.0 release). I'm fine with the name change. I would like to delay
the name change until a later point. This reverts the name change back to
the original.
CC: @SpriteOvO
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@sifive.com>
Use less resource intensive options for building nightly releases of CIRCT
full. The GitHub runners will run out of disk space if using
"relwithdebinfo" and/or "asserts=on". Use "release" and "asserts=off".
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@sifive.com>
Remove an explicit step in nightly and short integration tests that print
the ccache stats. This is already, conveniently, printing by the action
as part of the "Post ccache" step.
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@sifive.com>
Change the uploadFirrtlReleaseArtifacts workflow to use a reusable
workflow for building, testing, and installing firtool.
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@sifive.com>
Add a work-in-progress reusable GitHub Action to do a unified build, test,
and install of CIRCT. This is intended to be a replacement for pieces of
other GitHub actions which are doing similar, but slightly different
build/test/install.
This is directly landed onto main so that I can test this with a
workflow_dispatch event (manual trigger). This even can only be used for
workflows that are on the default branch.
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@sifive.com>
Remove the "-DLLVM_OPTIMIZED_TABLEGEN=ON" flag from cmake options used to
build firtool nightly binaries. This flag causes build errors when
combined with the "-DLLVM_ENABLE_ASSERTIONS=ON" flag which is desired for
testing of nightly binary builds.
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@sifive.com>
Short integration tests are filling the cache. Hopefully not appending
the timestamp will replace rather than add new ones. Also attempt to
enable sharing cache between nightly and short.
With our custom ccache stuff, cache hits don't update the ccache. This
is problematic since a the ccache might have a high miss rate. Switching
back to the ccache action as I'm assuming it's smarter than that.
Upstream now has option for installing these components for use in downstream standalone builds:
llvm/llvm-project@0807986 .
Look for (and prefer) this approach, and update our LLVM builds that we install to use this.
Skip tests when creating nightly firtool binaries if using the "scheduled"
event. This is already getting tested with the nightly integration tests in a
much more robust way and it's pointless to duplicate the tests here.
This gets the number of build steps down to ~1400 and causes binaries to upload
in 25 minutes.
Note: getting ccache setup on this job (and across all jobs) would help with
likely avoid the need for skipping tests.
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@sifive.com>
The ccache action by default uses the current date and time as part of the key
for caching. The main problem with this is that it can create too many
different cache entries, and can eject entries we don't want to be ejected.
There isn't a reason to keep more than one copy of this cache around, so using
a consistent key will cause it to eject itself. I put the name of the job in
the key so that we can identify which entries belong to what jobs when we
examine the cache. This change also updates to a newer version of the ccache
action.
Add an environment option to skip CIRCT testing. This is intended to be
used if the previous job knows that tests don't need to be run (or for
testing of actions with a slightly faster turnaround).
Expose this option as a workflow_dispatch input to make this easier to test.
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@sifive.com>
Add nightly builds of `firtool` for Linux targets only. These builds
occur everyday at 0700 UTC (0000 PDT/2300 PST).
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@sifive.com>
Change the CI for building a release of firtool from using a split
LLVM/CIRCT build to a unified build. The latter should be substantially
faster.
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@sifive.com>
Change the firtool release CI to not use a hard-coded build matrix.
Refactor this into two jobs:
1. "choose-matrix" which constructs JSON blobs of what the paramters are
for the build matrix. These JSON blobs are constructed from the step
environment ("env") to aid readability.
2. "publish" which reads the JSON blobs to construct the build
matrix. (This operates basically the same as before.)
This is done to later enable more plasticity of this CI job for uses which
are not just release publishing, but also for nightly publihsing.
Signed-off-by: Schuyler Eldridge <schuyler.eldridge@sifive.com>
This change splits the python wheel upload into 2 stages (separate jobs). In the
first stage, wheels are built and uploaded simply as artifacts through the
native GitHub actions upload-artifact action. This stage runs on the desired
matrix of configurations (manylinux, macos, etc.) and on all event triggers. In
the second stage, the corresponding wheels are downloaded (download-artifact)
and uploaded to PyPI. This stage only runs on ubuntu and only for tags or
nightly build (i.e. skipping workflow_dispatch).
The reason for this change is documented in #5420.
* Add Windows (firtool.exe)
* Remove Ubuntu 18.04 (runner is deprecated)
* Name archives based on OS and architecture
* linux-x64, macos-x64, linux-x64
* Windows uses .zip, Linux and MacOS use .tar.gz
* Add sha256 hashes for archives
We mostly just want the installed copy of LLVM,
so extract the single build file required and nuke
the build directory before proceeding.
The build directory is not used after this point,
so we can reclaim the disk space it uses.
Currently, the MacOS jobs cannot succeed, but that shouldn't block the
Linux jobs. In general, it seems best for all the jobs that can
succeed to move forward and upload their wheels.
* Change Upload Binaries to Upload Firrtl Binaries
This vastly reduces the size of the uploaded binaries while still
serving the needs of Firrtl users.
* Don't build tools when building LLVM
They are not part of the binary release and are unnecessary for building
CIRCT.
* Use CIRCT_RELEASE_TAG=firtool
This works for both tagged releases and will give a version derived from
`git describe --tags` when building from non-tagged commits.
Our llvm-lit arguments default to `-sv`, which makes the output succinct
and nice for interactive everyday use. However, in CI, we likely want to
see a more traditional output such that we can easily observe if tests
get stuck.
Add a `LLVM_LIT_ARGS` define to all CI cmake invocations that passes
just `-v` instead of `-sv`, to make lit output one line per tests that
is easily observable in the CI logs.
The workflow uses cibuildwheel to build the wheels, and uploads them
to GitHub. They're added to the artifacts for manual runs, and added
to the release assets when triggered by a release.
Currently, manylinux wheels are built for CPython 3.7 and 3.10.
This also updates the packaging scripts to include setuptools_scm to
take version numbers from Git tags.
* Disable ZSTD so that MacOS binaries do not dynamically link against
libzstd
* Use install target and package that to have include and lib
directories (in addition to the bin directory)
* Upload artifacts to the Github Actions page when this workflow is
triggered so that the artifacts can be checked
Enable the binary workflow to be run manually, but only upload binaries if it's a tag. I am not sure what happens if they're already present, so probably best to use this wisely.
While visiting, run the basic lit tests for LLVM/MLIR before bundling them for others (and sometimes ourselves) to use.
Drop cache as well: partly to avoid increased pressure on the cache from binaries/components built to run the tests that we don't presently, but also because I'm not sure the cached LLVM will likely ever be a cache hit since we release and update LLVM at about the same frequency. In the future, this may be useful if we did do more frequent releases, or ever promoted a pre-release to a an actual one.
cc #4473.
This PR swaps shared lib configuration of gcc+debug and clang+release CI so that we can reduce the binary size.
We just wanted to cover both static and shared lib builds so it should be fine to swap them.
These debug symbols take up too much disk space and prevent us from
running in the CI container. I enabled debug symbols a couple days ago,
and the issue has surfaced once LLVM was bumped. This still builds
CIRCT in RelWithDebInfo mode.
Building with debug symbols makes it much easier to debug a crash in CI,
which is sometimes necessary when you don't have access to a Windows
machine to reproduce on.
1. test: tweak test discovery to enable unittests, fix deps var.
Discovery will find the generated lit.site.cfg.py, which
knows how to load the lit.cfg.py.
This could be gated on whether we're building "standalone",
as flang does, but that still doesn't work as we are not part
of the config mapping.
cc #2750.
Also fixup typo in variable name, .._DEPS -> .._DEPENDS.
2. cmake: gtest is available if built with LLVM, set accordingly.
We only include the unittests directory if we think the GTEST bits
are available, which we were not indicating in the non-standalone case.
Fix this, and FWIW this is also done in flang's CMakeLists.txt .
3. [CI] Add check-circt-unit to testing.
GCC debug binaries are just too large and have been causing disk space issues intermittently. Just disable them for now as it is a longer effort to cut down on the amount of stuff we build. Fixes#3328 .
Release binaries produced by the `uploadBinaries` action now build with a
specified version computed using the tag of the release that caused the
job to run.
Don't cache entire llvm/build directory, we only need lit.
Also trim windows build's cached LLVM,
move to configuring from installed copies of LLVM/MLIR not build tree.
Before/After of cached LLVM data:
Windows: 1.15GB -> 478MB
clang: 241MB -> 63MB
gcc: 4.88GB -> 1.28GB
Cache limit is 10GB.
Has been silently failing for some time, pointed at non-existent
directory to find the compilation database.
While visiting, let clang-tidy operate in parallel as it can take
a small but non-trivial amount of time to process each file.
While trying to use the Mac firtool pre-built binary on a larger design
I noticed that it sometimes throws bogus errors, for example saying that
a value is not fully initialized when in fact it is, while the Linux
prebuilt binary works just fine on the same file. I did some testing and
determined that these errors appear in the Mac CIRCT build if using gcc
instead of clang. I'm not sure what the cause is -- I looked at the
build logs using gcc and noticed there were warnings about weak symbol
visibility during linking, but I'm not sure what these warnings really
mean.
The reason to use gcc over clang was to static link more libraries, but
clearly clang must be used on Mac. Let me know if you think we should
also use clang for the Linux build, but this PR just changes to clang
for Mac.
This PR switches the Mac build to use clang instead of gcc. Firtool now
looks like this:
```
$ otool -L firtool
firtool:
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1311.0.0)
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
/usr/local/opt/zstd/lib/libzstd.1.dylib (compatibility version 1.0.0, current version 1.5.2)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1200.3.0)
```
This PR also switches the Linux builds over to clang as well because it
looks like an upstream commit in LLVM broke the CIRCT build using gcc on
Ubuntu 18.04 (not 20.04 though). See here for details:
https://github.com/zyedidia/circt/runs/7799272570?check_suite_focus=true.
This commit introduces the base classes and utilities to implement
emission patterns and an emission printer which takes the role of a
driver that can be called in a circt-translate pass. Also includes
emission patterns for the builtin module and SCModule operations to
(1) allow to implement some basic unit and integration tests
(2) show the reviewer how emission patterns look like using this
infrastructure.
In order to be easily usable with Chisel soon, CIRCT should provide
pre-built binaries at least for macOS and Linux. Currently pre-built
Linux binaries are manually uploaded whenever a new build is made. This
PR adds a Github Actions script that builds and uploads macOS and Linux
binaries whenever a release (or pre-release) is made (requested in
#2842). The build from scratch takes approximately 2 hours.
The generated binaries are not fully static but are portable enough to
run on base versions of macOS and Ubuntu/other Linuxes. The dynamic
linking ends up looking like this:
Linux:
```
$ ldd firtool
linux-vdso.so.1 (0x00007ffc10fd1000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f19d67c3000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f19d6441000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f19d6229000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f19d5e64000)
/lib64/ld-linux-x86-64.so.2 (0x00007f19d7142000)
```
Mac:
```
$ otool -L firtool
firtool:
/usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1311.0.0)
```
The build is currently set up to use gcc/g++ instead of clang because
clang includes additional dynamic dependencies on Linux (`librt.so.1`,
`libdl.so.2`, and `libz.so.1`) and does not support the
`-static-libstdc++` flag on mac. The build currently runs on macOS-11
and Ubuntu-20.04.
Currently the uploaded package contains everything in the CIRCT `bin`
directory after a build.
Since we are now compiling separate LLVMs for GCC and clang, we should parallelize the CIRCT builds as well. Ditching the prebuild-and-cache-llvm step since that broke and nobody noticed.
Code compiled with Clang cannot be linked with GCC due to incompatibilities related to lambda closure capture lists.
This patch builds and caches LLVM with Clang to be used with the Clang Release build and with GCC to be used with the GCC Debug build of CIRCT.
The ABI issue and a potential fix is being tracked here:
https://github.com/itanium-cxx-abi/cxx-abi/issues/141
* LLVM bump (as we know it)
* minor format fix
* DCMAKE_BUILD_TYPE=?
* Update lib/Dialect/MSFT/MSFTOps.cpp
Co-authored-by: Andrew Young <youngar17@gmail.com>
* Update lib/Dialect/FIRRTL/FIRRTLOps.cpp
Co-authored-by: Andrew Young <youngar17@gmail.com>
* Update lib/Dialect/Calyx/CalyxOps.cpp
Co-authored-by: Andrew Young <youngar17@gmail.com>
* removed anon module test
Co-authored-by: Andrew Young <youngar17@gmail.com>
Hideto Ueno