replica
/
hqjenny-chipyard
2
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Merge pull request #82 from ucb-bar/ci-setup 

CI for Rebar
This commit is contained in:
Abraham Gonzalez 2019-05-15 15:57:35 -07:00 committed by GitHub
parent 86e2e5c458 deccae4959
commit a7822e1190
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
11 changed files with 614 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

54
.circleci/README.md Normal file
View File

@ -0,0 +1,54 @@
REBAR CI
========
Website: https://circleci.com/gh/ucb-bar/project-template
CircleCI Brief Explanation
---------------------------
CircleCI is controlled by the `config.yml` script.
It consists of a *workflow* which has a series of *jobs* within it that do particular tasks.
All jobs in the workflow must pass for the CI run to be successful.
At the bottom of the `config.yml` there is a `workflows:` section that specifies the order in which the jobs of the workflow should run.
For example:
- prepare-rocketchip:
requires:
- install-riscv-toolchain
- install-verilator
This specifies that the `prepare-rocketchip` job needs the `install-riscv-toolchain` and `install-verilator` steps to run before it can run.
All jobs in the CI workflow are specified at the top of `config.yml`
They specify a docker image to use (in this case a riscv-boom image since that is already available and works nicely) and an environment.
Finally, in the `steps:` section, the steps are run sequentially and state persists throughout a job.
So when you run something like `checkout` the next step has the checked out code.
Caching in the job is done by giving a file to cache on.
`restore_cache:` loads the cache into the environment if the key matches while `save_cache:` writes to the cache with the key IF IT IS NOT PRESENT.
Note, if the cache is already present for that key, the write to it is ignored.
Here the key is built from a string where the `checksum` portion converts the file given into a hash.
.circleci directory
-------------------
This directory contains all the collateral for the REBAR CI to work.
The following is included:
build-toolchains.sh # build either riscv-tools or esp-tools
build-verilator.sh # build verilator
create-hash.sh # create hashes of riscv-tools/esp-tools so circleci caching can work
do-rtl-build.sh # use verilator to build a sim executable
config.yml # main circleci config script to enumerate jobs/workflows
How things are setup for REBAR
------------------------------
The steps for CI to run are as follows.
1st, build the toolchains in parallel (note: `esp-tools` is currently not used in the run).
The docker image sets up the `PATH` and `RISCV` variable so that `riscv-tools` is the default (currently the `env.sh` script that is created at tool build is unused).
2nd, install verilator using the `*.mk` to cache unique versions of verilator (mainly for if verilator is bumped).
3rd, create the simulator binary.
This requires the `riscv-tools` for `fesvr` and `verilator` to be able to build the binary.
This stores all collateral for the tests (srcs, generated-srcs, sim binary, etc) to run "out of the gate" in the next job (make needs everything or else it will run again).
4th, finally run the tests that were wanted.

15
.circleci/build-toolchains.sh Executable file
View File

@ -0,0 +1,15 @@
#!/bin/bash
# create the riscv tools/esp tools binaries
# passed in as <riscv-tools or esp-tools>
# turn echo on and error on earliest command
set -ex
if [ ! -d "$HOME/$1-install" ]; then
cd $HOME/
# init all submodules including the tools
REBAR_DIR=$HOME/project ./project/scripts/build-toolchains.sh $1
fi

15
.circleci/build-verilator.sh Executable file
View File

@ -0,0 +1,15 @@
#!/bin/bash
# build verilator
# turn echo on and error on earliest command
set -ex
cd $HOME/project
cd sims/verisim
if [ ! -d "$HOME/project/sims/verisim/verilator" ]; then
# make verilator
make verilator_install
fi

473
.circleci/config.yml Normal file
View File

@ -0,0 +1,473 @@
# CircleCI Configuration File
# version of circleci
version: 2
# set of jobs to run
jobs:
install-riscv-toolchain:
docker:
- image: riscvboom/riscvboom-images:0.0.5
environment:
JVM_OPTS: -Xmx3200m # Customize the JVM maximum heap limit
TERM: dumb
steps:
# Checkout the code
- checkout
- run:
name: Create hash of toolchains
command: |
.circleci/create-hash.sh
- restore_cache:
keys:
- riscv-tools-installed-v1-{{ checksum "../riscv-tools.hash" }}
- run:
name: Building riscv-tools toolchain
command: |
.circleci/build-toolchains.sh riscv-tools
no_output_timeout: 120m
- save_cache:
key: riscv-tools-installed-v1-{{ checksum "../riscv-tools.hash" }}
paths:
- "/home/riscvuser/riscv-tools-install"
install-esp-toolchain:
docker:
- image: riscvboom/riscvboom-images:0.0.5
environment:
JVM_OPTS: -Xmx3200m # Customize the JVM maximum heap limit
TERM: dumb
steps:
# Checkout the code
- checkout
- run:
name: Create hash of toolchains
command: |
.circleci/create-hash.sh
- restore_cache:
keys:
- esp-tools-installed-v1-{{ checksum "../esp-tools.hash" }}
- run:
name: Building esp-tools toolchain
command: |
.circleci/build-toolchains.sh esp-tools
no_output_timeout: 120m
- save_cache:
key: esp-tools-installed-v1-{{ checksum "../esp-tools.hash" }}
paths:
- "/home/riscvuser/esp-tools-install"
install-verilator:
docker:
- image: riscvboom/riscvboom-images:0.0.5
environment:
JVM_OPTS: -Xmx3200m # Customize the JVM maximum heap limit
TERM: dumb
steps:
# Checkout the code
- checkout
- restore_cache:
keys:
- verilator-installed-v1-{{ checksum "sims/verisim/verilator.mk" }}
- run:
name: Build Verilator
command: |
.circleci/build-verilator.sh
no_output_timeout: 120m
- save_cache:
key: verilator-installed-v1-{{ checksum "sims/verisim/verilator.mk" }}
paths:
- "/home/riscvuser/project/sims/verisim/verilator"
prepare-example:
docker:
- image: riscvboom/riscvboom-images:0.0.5
environment:
JVM_OPTS: -Xmx3200m # Customize the JVM maximum heap limit
TERM: dumb
steps:
# Checkout the code
- checkout
- run:
name: Create hash of toolchains
command: |
.circleci/create-hash.sh
- restore_cache:
keys:
- riscv-tools-installed-v1-{{ checksum "../riscv-tools.hash" }}
- restore_cache:
keys:
- verilator-installed-v1-{{ checksum "sims/verisim/verilator.mk" }}
- run:
name: Building the example subproject using Verilator
command: .circleci/do-rtl-build.sh SUB_PROJECT=example
no_output_timeout: 120m
- save_cache:
key: example-{{ .Branch }}-{{ .Revision }}
paths:
- "/home/riscvuser/project"
prepare-boomexample:
docker:
- image: riscvboom/riscvboom-images:0.0.5
environment:
JVM_OPTS: -Xmx3200m # Customize the JVM maximum heap limit
TERM: dumb
steps:
# Checkout the code
- checkout
- run:
name: Create hash of toolchains
command: |
.circleci/create-hash.sh
- restore_cache:
keys:
- riscv-tools-installed-v1-{{ checksum "../riscv-tools.hash" }}
- restore_cache:
keys:
- verilator-installed-v1-{{ checksum "sims/verisim/verilator.mk" }}
- run:
name: Building the boomexample subproject using Verilator
command: .circleci/do-rtl-build.sh SUB_PROJECT=boomexample CONFIG=SmallDefaultBoomConfig
no_output_timeout: 120m
- save_cache:
key: boomexample-{{ .Branch }}-{{ .Revision }}
paths:
- "/home/riscvuser/project"
prepare-boom:
docker:
- image: riscvboom/riscvboom-images:0.0.5
environment:
JVM_OPTS: -Xmx3200m # Customize the JVM maximum heap limit
TERM: dumb
steps:
# Checkout the code
- checkout
- run:
name: Create hash of toolchains
command: |
.circleci/create-hash.sh
- restore_cache:
keys:
- riscv-tools-installed-v1-{{ checksum "../riscv-tools.hash" }}
- restore_cache:
keys:
- verilator-installed-v1-{{ checksum "sims/verisim/verilator.mk" }}
- run:
name: Building the boom subproject using Verilator
command: .circleci/do-rtl-build.sh SUB_PROJECT=boom
no_output_timeout: 120m
- save_cache:
key: boom-{{ .Branch }}-{{ .Revision }}
paths:
- "/home/riscvuser/project"
prepare-rocketchip:
docker:
- image: riscvboom/riscvboom-images:0.0.5
environment:
JVM_OPTS: -Xmx3200m # Customize the JVM maximum heap limit
TERM: dumb
steps:
# Checkout the code
- checkout
- run:
name: Create hash of toolchains
command: |
.circleci/create-hash.sh
- restore_cache:
keys:
- riscv-tools-installed-v1-{{ checksum "../riscv-tools.hash" }}
- restore_cache:
keys:
- verilator-installed-v1-{{ checksum "sims/verisim/verilator.mk" }}
- run:
name: Building the rocketchip subproject using Verilator
command: .circleci/do-rtl-build.sh SUB_PROJECT=rocketchip
no_output_timeout: 120m
- save_cache:
key: rocketchip-{{ .Branch }}-{{ .Revision }}
paths:
- "/home/riscvuser/project"
prepare-hwacha-verilog-only:
docker:
- image: riscvboom/riscvboom-images:0.0.5
environment:
JVM_OPTS: -Xmx3200m # Customize the JVM maximum heap limit
TERM: dumb
steps:
# Checkout the code
- checkout
- run:
name: Create hash of toolchains
command: |
.circleci/create-hash.sh
- restore_cache:
keys:
- riscv-tools-installed-v1-{{ checksum "../riscv-tools.hash" }}
- restore_cache:
keys:
- verilator-installed-v1-{{ checksum "sims/verisim/verilator.mk" }}
- run:
name: Building the hwacha subproject using Verilator
command: .circleci/do-rtl-build.sh SUB_PROJECT=hwacha verilog
no_output_timeout: 120m
- save_cache:
key: hwacha-{{ .Branch }}-{{ .Revision }}
paths:
- "/home/riscvuser/project"
example-run-benchmark-tests:
docker:
- image: riscvboom/riscvboom-images:0.0.5
environment:
JVM_OPTS: -Xmx3200m # Customize the JVM maximum heap limit
TERM: dumb
steps:
# Checkout the code
- checkout
- run:
name: Create hash of toolchains
command: |
.circleci/create-hash.sh
- restore_cache:
keys:
- riscv-tools-installed-v1-{{ checksum "../riscv-tools.hash" }}
- restore_cache:
keys:
- example-{{ .Branch }}-{{ .Revision }}
- run:
name: Run example benchmark tests
command: make run-bmark-tests -C sims/verisim SUB_PROJECT=example
boomexample-run-benchmark-tests:
docker:
- image: riscvboom/riscvboom-images:0.0.5
environment:
JVM_OPTS: -Xmx3200m # Customize the JVM maximum heap limit
TERM: dumb
steps:
# Checkout the code
- checkout
- run:
name: Create hash of toolchains
command: |
.circleci/create-hash.sh
- restore_cache:
keys:
- riscv-tools-installed-v1-{{ checksum "../riscv-tools.hash" }}
- restore_cache:
keys:
- boomexample-{{ .Branch }}-{{ .Revision }}
- run:
name: Run boomexample benchmark tests
command: make run-bmark-tests -C sims/verisim SUB_PROJECT=boomexample CONFIG=SmallDefaultBoomConfig
boom-run-benchmark-tests:
docker:
- image: riscvboom/riscvboom-images:0.0.5
environment:
JVM_OPTS: -Xmx3200m # Customize the JVM maximum heap limit
TERM: dumb
steps:
# Checkout the code
- checkout
- run:
name: Create hash of toolchains
command: |
.circleci/create-hash.sh
- restore_cache:
keys:
- riscv-tools-installed-v1-{{ checksum "../riscv-tools.hash" }}
- restore_cache:
keys:
- boom-{{ .Branch }}-{{ .Revision }}
- run:
name: Run boom benchmark tests
command: make run-bmark-tests -C sims/verisim SUB_PROJECT=boom
rocketchip-run-benchmark-tests:
docker:
- image: riscvboom/riscvboom-images:0.0.5
environment:
JVM_OPTS: -Xmx3200m # Customize the JVM maximum heap limit
TERM: dumb
steps:
# Checkout the code
- checkout
- run:
name: Create hash of toolchains
command: |
.circleci/create-hash.sh
- restore_cache:
keys:
- riscv-tools-installed-v1-{{ checksum "../riscv-tools.hash" }}
- restore_cache:
keys:
- rocketchip-{{ .Branch }}-{{ .Revision }}
- run:
name: Run rocketchip benchmark tests
command: make run-bmark-tests -C sims/verisim SUB_PROJECT=rocketchip
# hwacha-run-benchmark-tests:
# docker:
# - image: riscvboom/riscvboom-images:0.0.5
# environment:
# JVM_OPTS: -Xmx3200m # Customize the JVM maximum heap limit
# TERM: dumb
#
# steps:
# # Checkout the code
# - checkout
#
# - run:
# name: Create hash of toolchains
# command: |
# .circleci/create-hash.sh
#
# - restore_cache:
# keys:
# - riscv-tools-installed-v1-{{ checksum "../riscv-tools.hash" }}
#
# - restore_cache:
# keys:
# - hwacha-{{ .Branch }}-{{ .Revision }}
#
# - run:
# name: Run hwacha benchmark tests
# command: make run-bmark-tests -C sims/verisim SUB_PROJECT=hwacha
# Order and dependencies of jobs to run
workflows:
version: 2
build-and-test-rebar-integration:
jobs:
# Make the toolchains
- install-riscv-toolchain
- install-esp-toolchain
# Build verilator
- install-verilator
# Prepare the verilator builds
- prepare-example:
requires:
- install-riscv-toolchain
- install-verilator
- prepare-boomexample:
requires:
- install-riscv-toolchain
- install-verilator
- prepare-boom:
requires:
- install-riscv-toolchain
- install-verilator
- prepare-rocketchip:
requires:
- install-riscv-toolchain
- install-verilator
- prepare-hwacha-verilog-only:
requires:
- install-riscv-toolchain # TODO: Remove when esp-tools is used
- install-esp-toolchain
- install-verilator
# Run the respective tests
# Run the example tests
- example-run-benchmark-tests:
requires:
- install-riscv-toolchain
- prepare-example
- boomexample-run-benchmark-tests:
requires:
- install-riscv-toolchain
- prepare-boomexample
- boom-run-benchmark-tests:
requires:
- install-riscv-toolchain
- prepare-boom
- rocketchip-run-benchmark-tests:
requires:
- install-riscv-toolchain
- prepare-rocketchip
# - hwacha-run-benchmark-tests:
# requires:
# - install-riscv-toolchain # TODO: Remove when esp-tools is used
# - install-esp-toolchain
# - prepare-hwacha

17
.circleci/create-hash.sh Executable file
View File

@ -0,0 +1,17 @@
#!/bin/bash
# get the hash of riscv-tools
# turn echo on and error on earliest command
set -ex
# enter bhd repo
cd $HOME/project
# get the version of riscv-tools from the git submodule hash
git submodule status | grep "riscv-tools" | awk '{print$1}' | grep -o "[[:alnum:]]*" >> $HOME/riscv-tools.hash
git submodule status | grep "esp-tools" | awk '{print$1}' | grep -o "[[:alnum:]]*" >> $HOME/esp-tools.hash
echo "Hashfile for riscv-tools and esp-tools created in $HOME"
echo "Contents: riscv-tools:$(cat $HOME/riscv-tools.hash)"
echo "Contents: esp-tools:$(cat $HOME/esp-tools.hash)"

20
.circleci/do-rtl-build.sh Executable file
View File

@ -0,0 +1,20 @@
#!/bin/bash
# create the different verilator builds
# argument is the make command string
# turn echo on and error on earliest command
set -ex
# init all submodules
cd $HOME/project
./scripts/init-submodules-no-riscv-tools.sh
# enter the verisim directory and build the specific config
cd sims/verisim
make clean
# run the particular build command
make JAVA_ARGS="-Xmx2G -Xss8M" $@
rm -rf ../../project

2
README.md
View File

@ -1,4 +1,4 @@
# RISC-V Project Template
# RISC-V Project Template [![CircleCI](https://circleci.com/gh/ucb-bar/project-template/tree/master.svg?style=svg)](https://circleci.com/gh/ucb-bar/project-template/tree/master)
**This branch is under development**
**It currently has many submodules**

2
generators/example/src/main/scala/ConfigMixins.scala
View File

@ -5,7 +5,7 @@ import freechips.rocketchip.config.{Parameters, Config}
import freechips.rocketchip.subsystem.{WithRoccExample, WithNMemoryChannels, WithNBigCores, WithRV32}
import freechips.rocketchip.diplomacy.{LazyModule, ValName}
import freechips.rocketchip.devices.tilelink.BootROMParams
import freechips.rocketchip.tile.XLen
import freechips.rocketchip.tile.{XLen}
import testchipip._
import sifive.blocks.devices.gpio._

10
generators/example/src/main/scala/Configs.scala
View File

@ -2,7 +2,7 @@ package example
import chisel3._
import freechips.rocketchip.config.{Config}
import freechips.rocketchip.subsystem.{WithRoccExample, WithNMemoryChannels, WithNBigCores, WithRV32, WithExtMemSize}
import freechips.rocketchip.subsystem.{WithRoccExample, WithNMemoryChannels, WithNBigCores, WithRV32, WithExtMemSize, WithNBanks}
import testchipip._
// --------------
@ -68,10 +68,18 @@ class BaseBoomConfig extends Config(
new WithBootROM ++
new boom.system.BoomConfig)
class SmallBaseBoomConfig extends Config(
new WithBootROM ++
new boom.system.SmallBoomConfig)
class DefaultBoomConfig extends Config(
new WithNormalBoomTop ++
new BaseBoomConfig)
class SmallDefaultBoomConfig extends Config(
new WithNormalBoomTop ++
new SmallBaseBoomConfig)
class HwachaBoomConfig extends Config(
new hwacha.DefaultHwachaConfig ++
new DefaultBoomConfig)

5
scripts/build-toolchains.sh
View File

@ -6,6 +6,7 @@ set -o pipefail
unamestr=$(uname)
RDIR=$(pwd)
: ${REBAR_DIR:=$(pwd)} #default value is the PWD unless overridden
if [ $# -ne 0 ]; then
TOOLCHAIN=$1
@ -25,8 +26,8 @@ RISCV="$(pwd)/$INSTALL_DIR"
# install risc-v tools
export RISCV="$RISCV"
git submodule update --init --recursive toolchains/$TOOLCHAIN #--jobs 8
cd "toolchains/$TOOLCHAIN"
git -C $REBAR_DIR submodule update --init --recursive toolchains/$TOOLCHAIN #--jobs 8
cd "$REBAR_DIR/toolchains/$TOOLCHAIN"
export MAKEFLAGS="-j16"
./build.sh
cd $RDIR

7
variables.mk
View File

@ -122,13 +122,18 @@ sim_dotf ?= $(build_dir)/sim_files.f
sim_harness_blackboxes ?= $(build_dir)/firrtl_black_box_resource_files.harness.f
sim_top_blackboxes ?= $(build_dir)/firrtl_black_box_resource_files.top.f
#########################################################################################
# java arguments used in sbt
#########################################################################################
JAVA_ARGS ?= -Xmx8G -Xss8M -XX:MaxPermSize=256M
#########################################################################################
# default sbt launch command
#########################################################################################
SCALA_VERSION=2.12.4
SCALA_VERSION_MAJOR=$(basename $(SCALA_VERSION))
SBT ?= java -Xmx2G -Xss8M -XX:MaxPermSize=256M -jar $(ROCKETCHIP_DIR)/sbt-launch.jar ++$(SCALA_VERSION)
SBT ?= java $(JAVA_ARGS) -jar $(ROCKETCHIP_DIR)/sbt-launch.jar ++$(SCALA_VERSION)
#########################################################################################
# output directory for tests