wu-arch/chipyard
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Ariane Integration (#448)

Browse Source 
* [ariane/make] integrate ariane | have verilator be installed on path not in makefile

* [misc] warn on verilator not found | search for v files | cleanup build.sbt + .gitignore

* [firesim] bump

* [ci] add midas ariane tests

* [docker/ci] use new docker-image with verilator | re-elab on v changes for ariane | address comments

* [ci] remove references to local verilator install

* [verilator] update flags

* [verilator] minimal set of flags for ariane

* [ariane] bump ariane to master

* [ci] revert to 4.016 verilator

* [ci] install verilator to ci server | misc compile fixes

* [ci/make] add longer ci timeout | update when assert is added in verilator sim

* [firesim] bump for misc. updates

* [make/ci] cleanup makefile and remove firesim tests of it

* [docs/firesim] bump and clean docs

* [firesim] bump

* [ci] use remote verilator for midas tests

* [misc] cleanup built.sbt more

* [firesim] bump

* [misc] bump build.sbt patch for tutorials

* [firesim/ci] cleanup and bump firesim
This commit is contained in:
Abraham Gonzalez
2020-03-09 18:06:41 -07:00
committed by GitHub
parent 7cf37b604e
commit d0bec3fba7
34 changed files with 315 additions and 202 deletions
Download Patch File Download Diff File Expand all files Collapse all files

9
.circleci/README.md
View File

@@ -16,9 +16,8 @@ For example:
- prepare-rocketchip:
requires:
- install-riscv-toolchain
- install-verilator
This specifies that the `prepare-rocketchip` job needs the `install-riscv-toolchain` and `install-verilator` steps to run before it can run.
This specifies that the `prepare-rocketchip` job needs the `install-riscv-toolchain` steps to run before it can run.
All jobs in the CI workflow are specified at the top of `config.yml`
They specify a docker image to use (in this case a riscv-boom image since that is already available and works nicely) and an environment.
@@ -36,7 +35,6 @@ This directory contains all the collateral for the Chipyard CI to work.
The following is included:
`build-toolchains.sh` # build either riscv-tools or esp-tools
`build-verilator.sh` # build verilator (remotely)
`create-hash.sh` # create hashes of riscv-tools/esp-tools so circleci caching can work
`do-rtl-build.sh` # use verilator to build a sim executable (remotely)
`config.yml` # main circleci config script to enumerate jobs/workflows
@@ -48,11 +46,10 @@ How things are setup for Chipyard
The steps for CI to run are as follows.
1st, build the toolchains in parallel (note: `esp-tools` is currently not used in the run).
The docker image sets up the `PATH` and `RISCV` variable so that `riscv-tools` is the default (currently the `env.sh` script that is created at tool build is unused).
2nd, install verilator using the `*.mk` to cache unique versions of verilator (mainly for if verilator is bumped).
3rd, create the simulator binary.
2nd, create the simulator binary.
This requires the `riscv-tools` for `fesvr` and `verilator` to be able to build the binary.
This stores all collateral for the tests (srcs, generated-srcs, sim binary, etc) to run "out of the gate" in the next job (make needs everything or else it will run again).
4th, finally run the tests that were wanted.
3rd, finally run the desired tests.
Other CI Setup
--------------

35
.circleci/build-verilator.sh
View File

@@ -1,35 +0,0 @@
#!/bin/bash
# build verilator
# turn echo on and error on earliest command
set -ex
# get shared variables
SCRIPT_DIR="$( cd "$( dirname "$0" )" && pwd )"
source $SCRIPT_DIR/defaults.sh
# call clean on exit
trap clean EXIT
run_script $LOCAL_CHIPYARD_DIR/.circleci/clean-old-files.sh $CI_DIR
if [ ! -d "$LOCAL_VERILATOR_DIR" ]; then
# set stricthostkeychecking to no (must happen before rsync)
run "echo \"Ping $SERVER\""
clean
run "mkdir -p $REMOTE_CHIPYARD_DIR"
copy $LOCAL_CHIPYARD_DIR/ $SERVER:$REMOTE_CHIPYARD_DIR
run "make -j$NPROC -C $REMOTE_SIM_DIR VERILATOR_INSTALL_DIR=$REMOTE_VERILATOR_DIR verilator_install"
# copy so that circleci can cache
mkdir -p $LOCAL_CHIPYARD_DIR
mkdir -p $LOCAL_VERILATOR_DIR
copy $SERVER:$REMOTE_CHIPYARD_DIR/ $LOCAL_CHIPYARD_DIR
copy $SERVER:$REMOTE_VERILATOR_DIR/ $LOCAL_VERILATOR_DIR
cp -r $LOCAL_VERILATOR_DIR/install/bin/* $LOCAL_VERILATOR_DIR/install/share/verilator/bin/.
fi

56
.circleci/config.yml
View File

@@ -4,9 +4,6 @@
version: 2.1
parameters:
verilator-cache-version:
type: string
default: "v3"
tools-cache-version:
type: string
default: "v4"
@@ -15,7 +12,7 @@ parameters:
executors:
main-env:
docker:
- image: riscvboom/riscvboom-images:0.0.12
- image: riscvboom/riscvboom-images:0.0.13
environment:
JVM_OPTS: -Xmx3200m # Customize the JVM maximum heap limit
@@ -54,8 +51,8 @@ commands:
- "3e:c3:02:5b:ed:64:8c:b7:b0:04:43:bc:83:43:73:1e"
- checkout
setup-tools-verilator:
description: "Get toolchain and verilator"
setup-tools:
description: "Get toolchain"
parameters:
tools-version:
type: string
@@ -69,9 +66,6 @@ commands:
- restore_cache:
keys:
- << parameters.tools-version >>-installed-<< pipeline.parameters.tools-cache-version >>-{{ checksum "../<< parameters.tools-version >>.hash" }}
- restore_cache:
keys:
- verilator-installed-<< pipeline.parameters.verilator-cache-version >>-{{ checksum "sims/verilator/verilator.mk" }}
prepare-rtl:
description: "Run the prepare step of RTL"
@@ -88,7 +82,7 @@ commands:
type: string
default: "do-rtl-build.sh"
steps:
- setup-tools-verilator:
- setup-tools:
tools-version: "<< parameters.tools-version >>"
- run:
name: Building << parameters.project-key >> subproject using Verilator
@@ -117,7 +111,7 @@ commands:
type: string
default: "10m"
steps:
- setup-tools-verilator:
- setup-tools:
tools-version: "<< parameters.tools-version >>"
- restore_cache:
keys:
@@ -166,18 +160,10 @@ jobs:
executor: main-env
steps:
- ssh-checkout
- restore_cache:
keys:
- verilator-installed-<< pipeline.parameters.verilator-cache-version >>-{{ checksum "sims/verilator/verilator.mk" }}
- run:
name: Build Verilator
name: Install Verilator to remote
command: |
.circleci/build-verilator.sh
no_output_timeout: 120m
- save_cache:
key: verilator-installed-<< pipeline.parameters.verilator-cache-version >>-{{ checksum "sims/verilator/verilator.mk" }}
paths:
- "/home/riscvuser/verilator"
.circleci/install-verilator.sh
build-extra-tests:
executor: main-env
steps:
@@ -262,9 +248,11 @@ jobs:
- prepare-rtl:
project-key: "fireboom"
build-script: "do-firesim-build.sh"
prepare-chipyard-ariane:
executor: main-env
steps:
- prepare-rtl:
project-key: "chipyard-ariane"
chipyard-rocket-run-tests:
executor: main-env
steps:
@@ -327,14 +315,19 @@ jobs:
project-key: "fireboom"
run-script: "run-firesim-tests.sh"
timeout: "30m"
midasexamples-run-tests:
executor: main-env
steps:
- setup-tools-verilator
- setup-tools
- run:
name: Run midasexamples tests
command: .circleci/run-midasexamples-tests.sh
chipyard-ariane-run-tests:
executor: main-env
steps:
- run-tests:
project-key: "chipyard-ariane"
# Order and dependencies of jobs to run
workflows:
version: 2
@@ -357,7 +350,6 @@ workflows:
- install-esp-toolchain
# Build verilator
- install-verilator
- commit-on-master-check
@@ -431,13 +423,17 @@ workflows:
- install-riscv-toolchain
- install-verilator
- prepare-chipyard-ariane:
requires:
- install-riscv-toolchain
- install-verilator
# Run the respective tests
# Run midasexamples test
- midasexamples-run-tests:
requires:
- install-riscv-toolchain
- install-verilator
# Run the example tests
- chipyard-rocket-run-tests:
@@ -486,3 +482,7 @@ workflows:
requires:
- prepare-fireboom
- build-extra-tests
- chipyard-ariane-run-tests:
requires:
- prepare-chipyard-ariane

8
.circleci/defaults.sh
View File

@@ -20,22 +20,24 @@ clean () {
# make parallelism
NPROC=8
# verilator version
VERILATOR_VERSION=v4.028
# remote variables
REMOTE_WORK_DIR=$CI_DIR/$CIRCLE_PROJECT_REPONAME-$CIRCLE_BRANCH-$CIRCLE_SHA1-$CIRCLE_JOB
REMOTE_RISCV_DIR=$REMOTE_WORK_DIR/riscv-tools-install
REMOTE_ESP_DIR=$REMOTE_WORK_DIR/esp-tools-install
REMOTE_CHIPYARD_DIR=$REMOTE_WORK_DIR/chipyard
REMOTE_VERILATOR_DIR=$REMOTE_WORK_DIR/verilator
REMOTE_SIM_DIR=$REMOTE_CHIPYARD_DIR/sims/verilator
REMOTE_FIRESIM_DIR=$REMOTE_CHIPYARD_DIR/sims/firesim/sim
REMOTE_JAVA_ARGS="-Xmx8G -Xss8M -Dsbt.ivy.home=$REMOTE_WORK_DIR/.ivy2 -Dsbt.global.base=$REMOTE_WORK_DIR/.sbt -Dsbt.boot.directory=$REMOTE_WORK_DIR/.sbt/boot"
REMOTE_VERILATOR_DIR=$CI_DIR/$CIRCLE_PROJECT_REPONAME-$CIRCLE_BRANCH-$CIRCLE_SHA1-verilator-install
# local variables (aka within the docker container)
LOCAL_CHECKOUT_DIR=$HOME/project
LOCAL_RISCV_DIR=$HOME/riscv-tools-install
LOCAL_ESP_DIR=$HOME/esp-tools-install
LOCAL_CHIPYARD_DIR=$LOCAL_CHECKOUT_DIR
LOCAL_VERILATOR_DIR=$HOME/verilator
LOCAL_SIM_DIR=$LOCAL_CHIPYARD_DIR/sims/verilator
LOCAL_FIRESIM_DIR=$LOCAL_CHIPYARD_DIR/sims/firesim/sim
@@ -53,3 +55,5 @@ mapping["tracegen"]="SUB_PROJECT=chipyard CONFIG=NonBlockingTraceGenL2Config TOP
mapping["tracegen-boom"]="SUB_PROJECT=chipyard CONFIG=BoomTraceGenConfig TOP=TraceGenSystem"
mapping["firesim"]="DESIGN=FireSim TARGET_CONFIG=WithNIC_DDR3FRFCFSLLC4MB_FireSimRocketConfig PLATFORM_CONFIG=BaseF1Config"
mapping["fireboom"]="DESIGN=FireSim TARGET_CONFIG=WithNIC_DDR3FRFCFSLLC4MB_FireSimLargeBoomConfig PLATFORM_CONFIG=BaseF1Config"
mapping["chipyard-ariane"]="SUB_PROJECT=chipyard CONFIG=ArianeConfig"
mapping["fireariane"]="DESIGN=FireSim TARGET_CONFIG=WithNIC_DDR3FRFCFSLLC4MB_FireSimArianeConfig PLATFORM_CONFIG=BaseF1Config"

13
.circleci/do-firesim-build.sh
View File

@@ -28,18 +28,15 @@ run "echo \"Ping $SERVER\""
clean
# copy over riscv/esp-tools, verilator, and chipyard to remote
# copy over riscv/esp-tools, and chipyard to remote
run "mkdir -p $REMOTE_CHIPYARD_DIR"
run "mkdir -p $REMOTE_VERILATOR_DIR"
copy $LOCAL_CHIPYARD_DIR/ $SERVER:$REMOTE_CHIPYARD_DIR
copy $LOCAL_VERILATOR_DIR/ $SERVER:$REMOTE_VERILATOR_DIR
run "cp -r ~/.ivy2 $REMOTE_WORK_DIR"
run "cp -r ~/.sbt $REMOTE_WORK_DIR"
TOOLS_DIR=$REMOTE_RISCV_DIR
LD_LIB_DIR=$REMOTE_RISCV_DIR/lib
VERILATOR_BIN_DIR=$REMOTE_VERILATOR_DIR/install/bin
if [ $1 = "hwacha" ] || [ $1 = "gemmini" ]; then
TOOLS_DIR=$REMOTE_ESP_DIR
@@ -54,9 +51,11 @@ fi
# Build MIDAS-level verilator sim
FIRESIM_VARS="${mapping[$1]}"
run "export FIRESIM_ENV_SOURCED=1; make -C $REMOTE_FIRESIM_DIR clean"
run "export RISCV=\"$TOOLS_DIR\"; export LD_LIBRARY_PATH=\"$LD_LIB_DIR\"; \
export PATH=\"$VERILATOR_BIN_DIR:\$PATH\"; export FIRESIM_ENV_SOURCED=1; \
export VERILATOR_ROOT=$REMOTE_VERILATOR_DIR/install/share/verilator; \
run "export RISCV=\"$TOOLS_DIR\"; \
export LD_LIBRARY_PATH=\"$LD_LIB_DIR\"; \
export PATH=\"$REMOTE_VERILATOR_DIR/bin:\$PATH\"; \
export VERILATOR_ROOT=\"$REMOTE_VERILATOR_DIR\"; \
export FIRESIM_ENV_SOURCED=1; \
make -C $REMOTE_FIRESIM_DIR JAVA_ARGS=\"$REMOTE_JAVA_ARGS\" $FIRESIM_VARS verilator"
run "rm -rf $REMOTE_CHIPYARD_DIR/project"

12
.circleci/do-rtl-build.sh
View File

@@ -21,11 +21,9 @@ run "echo \"Ping $SERVER\""
clean
# copy over riscv/esp-tools, verilator, and chipyard to remote
# copy over riscv/esp-tools, and chipyard to remote
run "mkdir -p $REMOTE_CHIPYARD_DIR"
run "mkdir -p $REMOTE_VERILATOR_DIR"
copy $LOCAL_CHIPYARD_DIR/ $SERVER:$REMOTE_CHIPYARD_DIR
copy $LOCAL_VERILATOR_DIR/ $SERVER:$REMOTE_VERILATOR_DIR
run "cp -r ~/.ivy2 $REMOTE_WORK_DIR"
run "cp -r ~/.sbt $REMOTE_WORK_DIR"
@@ -56,9 +54,11 @@ fi
# enter the verilator directory and build the specific config on remote server
run "make -C $REMOTE_SIM_DIR clean"
run "export RISCV=\"$TOOLS_DIR\"; export LD_LIBRARY_PATH=\"$LD_LIB_DIR\"; \
export VERILATOR_ROOT=$REMOTE_VERILATOR_DIR/install/share/verilator; \
make -j$NPROC -C $REMOTE_SIM_DIR VERILATOR_INSTALL_DIR=$REMOTE_VERILATOR_DIR JAVA_ARGS=\"$REMOTE_JAVA_ARGS\" ${mapping[$1]}"
run "export RISCV=\"$TOOLS_DIR\"; \
export LD_LIBRARY_PATH=\"$LD_LIB_DIR\"; \
export PATH=\"$REMOTE_VERILATOR_DIR/bin:\$PATH\"; \
export VERILATOR_ROOT=\"$REMOTE_VERILATOR_DIR\"; \
make -j$NPROC -C $REMOTE_SIM_DIR JAVA_ARGS=\"$REMOTE_JAVA_ARGS\" ${mapping[$1]}"
run "rm -rf $REMOTE_CHIPYARD_DIR/project"
# copy back the final build

23
.circleci/install-verilator.sh Executable file
View File

@@ -0,0 +1,23 @@
#!/bin/bash
# move verilator to the remote server
# turn echo on and error on earliest command
set -ex
# get shared variables
SCRIPT_DIR="$( cd "$( dirname "$0" )" && pwd )"
source $SCRIPT_DIR/defaults.sh
run_script $LOCAL_CHIPYARD_DIR/.circleci/clean-old-files.sh $CI_DIR
# set stricthostkeychecking to no (must happen before rsync)
run "echo \"Ping $SERVER\""
run "git clone http://git.veripool.org/git/verilator $REMOTE_VERILATOR_DIR; \
cd $REMOTE_VERILATOR_DIR; \
git checkout $VERILATOR_VERSION; \
autoconf; \
export VERILATOR_ROOT=$REMOTE_VERILATOR_DIR; \
./configure; \
make -j$NPROC;"

1
.circleci/run-firesim-tests.sh
View File

@@ -7,7 +7,6 @@ set -ex
SCRIPT_DIR="$( cd "$( dirname "$0" )" && pwd )"
source $SCRIPT_DIR/defaults.sh
export PATH=$LOCAL_VERILATOR_DIR/install/bin:$PATH
export FIRESIM_ENV_SOURCED=1
SIMULATION_ARGS="${mapping[$1]}"

13
.circleci/run-midasexamples-tests.sh
View File

@@ -19,13 +19,11 @@ run "echo \"Ping $SERVER\""
clean
# copy over riscv-tools, verilator, and chipyard to remote
# copy over riscv-tools, and chipyard to remote
run "mkdir -p $REMOTE_CHIPYARD_DIR"
run "mkdir -p $REMOTE_VERILATOR_DIR"
run "mkdir -p $REMOTE_RISCV_DIR"
copy $LOCAL_CHIPYARD_DIR/ $SERVER:$REMOTE_CHIPYARD_DIR
copy $LOCAL_VERILATOR_DIR/ $SERVER:$REMOTE_VERILATOR_DIR
copy $LOCAL_RISCV_DIR/ $SERVER:$REMOTE_RISCV_DIR
# Copy ivy2 and sbt directories
@@ -35,12 +33,13 @@ run "cp -r ~/.sbt $REMOTE_WORK_DIR"
TOOLS_DIR=$REMOTE_RISCV_DIR
LD_LIB_DIR=$REMOTE_RISCV_DIR/lib
VERILATOR_BIN_DIR=$REMOTE_VERILATOR_DIR/install/bin
# Run midasexamples test
run "export FIRESIM_ENV_SOURCED=1; make -C $REMOTE_FIRESIM_DIR clean"
run "export RISCV=\"$TOOLS_DIR\"; export LD_LIBRARY_PATH=\"$LD_LIB_DIR\"; \
export PATH=\"$VERILATOR_BIN_DIR:\$PATH\"; export FIRESIM_ENV_SOURCED=1; \
export VERILATOR_ROOT=$REMOTE_VERILATOR_DIR/install/share/verilator; \
run "export RISCV=\"$TOOLS_DIR\"; \
export LD_LIBRARY_PATH=\"$LD_LIB_DIR\"; \
export FIRESIM_ENV_SOURCED=1; \
export PATH=\"$REMOTE_VERILATOR_DIR/bin:\$PATH\"; \
export VERILATOR_ROOT=\"$REMOTE_VERILATOR_DIR\"; \
make -C $REMOTE_FIRESIM_DIR JAVA_ARGS=\"$REMOTE_JAVA_ARGS\" TARGET_PROJECT=midasexamples test"

13
.circleci/run-tests.sh
View File

@@ -9,14 +9,12 @@ set -ex
SCRIPT_DIR="$( cd "$( dirname "$0" )" && pwd )"
source $SCRIPT_DIR/defaults.sh
export VERILATOR_ROOT=$LOCAL_VERILATOR_DIR/install/share/verilator
run_bmark () {
make run-bmark-tests-fast -j$NPROC -C $LOCAL_SIM_DIR VERILATOR_INSTALL_DIR=$LOCAL_VERILATOR_DIR $@
make run-bmark-tests-fast -j$NPROC -C $LOCAL_SIM_DIR $@
}
run_asm () {
make run-asm-tests-fast -j$NPROC -C $LOCAL_SIM_DIR VERILATOR_INSTALL_DIR=$LOCAL_VERILATOR_DIR $@
make run-asm-tests-fast -j$NPROC -C $LOCAL_SIM_DIR $@
}
run_both () {
@@ -25,7 +23,7 @@ run_both () {
}
run_tracegen () {
make tracegen -C $LOCAL_SIM_DIR VERILATOR_INSTALL_DIR=$LOCAL_VERILATOR_DIR $@
make tracegen -C $LOCAL_SIM_DIR $@
}
# TODO BUG: the run-binary command forces a rebuild of the simulator in CI
@@ -47,7 +45,7 @@ case $1 in
export RISCV=$LOCAL_ESP_DIR
export LD_LIBRARY_PATH=$LOCAL_ESP_DIR/lib
export PATH=$RISCV/bin:$PATH
make run-rv64uv-p-asm-tests -j$NPROC -C $LOCAL_SIM_DIR VERILATOR_INSTALL_DIR=$LOCAL_VERILATOR_DIR ${mapping[$1]}
make run-rv64uv-p-asm-tests -j$NPROC -C $LOCAL_SIM_DIR ${mapping[$1]}
;;
chipyard-gemmini)
export RISCV=$LOCAL_ESP_DIR
@@ -70,6 +68,9 @@ case $1 in
tracegen-boom)
run_tracegen ${mapping[$1]}
;;
chipyard-ariane)
make run-binary-fast -C $LOCAL_SIM_DIR ${mapping[$1]} BINARY=$RISCV/riscv64-unknown-elf/share/riscv-tests/benchmarks/dhrystone.riscv
;;
*)
echo "No set of tests for $1. Did you spell it right?"
exit 1