diff --git a/.circleci/README.md b/.circleci/README.md deleted file mode 100644 index 0c53405d..00000000 --- a/.circleci/README.md +++ /dev/null @@ -1,75 +0,0 @@ -Chipyard CI -=========== - -Website: https://circleci.com/gh/ucb-bar/chipyard - -CircleCI Brief Explanation ---------------------------- - -CircleCI is controlled by the `config.yml` script. -It consists of a *workflow* which has a series of *jobs* within it that do particular tasks. -All jobs in the workflow must pass for the CI run to be successful. - -At the bottom of the `config.yml` there is a `workflows:` section that specifies the order in which the jobs of the workflow should run. -For example: - - - prepare-rocketchip: - requires: - - install-riscv-toolchain - -This specifies that the `prepare-rocketchip` job needs the `install-riscv-toolchain` steps to run before it can run. - -All jobs in the CI workflow are specified at the top of `config.yml` -They specify a docker image to use (in this case a riscv-boom image since that is already available and works nicely) and an environment. -Finally, in the `steps:` section, the steps are run sequentially and state persists throughout a job. -So when you run something like `checkout` the next step has the checked out code. -Caching in the job is done by giving a file to cache on. -`restore_cache:` loads the cache into the environment if the key matches while `save_cache:` writes to the cache with the key IF IT IS NOT PRESENT. -Note, if the cache is already present for that key, the write to it is ignored. -Here the key is built from a string where the `checksum` portion converts the file given into a hash. - -.circleci directory -------------------- - -This directory contains all the collateral for the Chipyard CI to work. -The following is included: - - `build-toolchains.sh` # build either riscv-tools or esp-tools - `create-hash.sh` # create hashes of riscv-tools/esp-tools so circleci caching can work - `do-rtl-build.sh` # use verilator to build a sim executable (remotely) - `config.yml` # main circleci config script to enumerate jobs/workflows - `defaults.sh` # default variables used - `check-commit.sh` # check that submodule commits are valid - `build-extra-tests.sh` # build default chipyard tests located in tests/ - `clean-old-files.sh` # clean up build server files - `do-fpga-rtl-build.sh` # similar to `do-rtl-build` but using fpga/ - `install-verilator.sh` # install verilator on build server - `run-firesim-scala-tests.sh` # run firesim scala tests - `run-tests.sh # run tests for a specific set of designs - `images/` # docker image used in CI - -How things are setup for Chipyard ---------------------------------- - -The steps for CI to run are as follows. -1st, build the toolchains in parallel (note: `esp-tools` is currently not used in the run). -The docker image sets up the `PATH` and `RISCV` variable so that `riscv-tools` is the default (currently the `env.sh` script that is created at tool build is unused). -2nd, create the simulator binary. -This requires the `riscv-tools` for `fesvr` and `verilator` to be able to build the binary. -This stores all collateral for the tests (srcs, generated-srcs, sim binary, etc) to run "out of the gate" in the next job (make needs everything or else it will run again). -3rd, finally run the desired tests. - -Other CI Setup --------------- - -To get the CI to work correctly you need to setup CircleCI environment variables to point to the remote directory to build files and the server user/ip. -In the project settings, you can find this under "Build Settings" "Environment Variables". -You need to add two variables like the following: - -CI\_DIR = /path/to/where/you/want/to/store/remote/files -SERVER = username@myserver.coolmachine.berkeley.edu - -Additionally, you need to add under the "PERMISSIONS" "SSH Permissions" section a private key that is on the build server that you are using. -After adding a private key, it will show a fingerprint that should be added under the jobs that need to be run. - -Note: On the remote server you need to have the `*.pub` key file added to the `authorized_keys` file. diff --git a/.circleci/build-extra-tests.sh b/.circleci/build-extra-tests.sh deleted file mode 100755 index e38b50fe..00000000 --- a/.circleci/build-extra-tests.sh +++ /dev/null @@ -1,11 +0,0 @@ -#!/bin/bash - -# turn echo on and error on earliest command -set -ex - -# get shared variables -SCRIPT_DIR="$( cd "$( dirname "$0" )" && pwd )" -source $SCRIPT_DIR/defaults.sh - -make -C $LOCAL_CHIPYARD_DIR/tests clean -make -C $LOCAL_CHIPYARD_DIR/tests diff --git a/.circleci/build-toolchains.sh b/.circleci/build-toolchains.sh deleted file mode 100755 index 160b6f5a..00000000 --- a/.circleci/build-toolchains.sh +++ /dev/null @@ -1,18 +0,0 @@ -#!/bin/bash - -# create the riscv tools/esp tools binaries -# passed in as - -# turn echo on and error on earliest command -set -ex - -# get shared variables -SCRIPT_DIR="$( cd "$( dirname "$0" )" && pwd )" -source $SCRIPT_DIR/defaults.sh - -if [ ! -d "$HOME/$1-install" ]; then - cd $HOME - - # init all submodules including the tools - CHIPYARD_DIR="$LOCAL_CHIPYARD_DIR" NPROC=$CI_MAKE_NPROC $LOCAL_CHIPYARD_DIR/scripts/build-toolchains.sh $1 -fi diff --git a/.circleci/check-commit.sh b/.circleci/check-commit.sh deleted file mode 100755 index b52ff337..00000000 --- a/.circleci/check-commit.sh +++ /dev/null @@ -1,152 +0,0 @@ -#!/bin/bash - -# check to see that submodule commits are present on the master branch - -# turn echo on and error on earliest command -set -ex - -# get shared variables -SCRIPT_DIR="$( cd "$( dirname "$0" )" && pwd )" -source $SCRIPT_DIR/defaults.sh - -# enter bhd repo -cd $LOCAL_CHIPYARD_DIR - -# ignore the private vlsi submodules -git config submodule.vlsi/hammer-cadence-plugins.update none -git config submodule.vlsi/hammer-mentor-plugins.update none -git config submodule.vlsi/hammer-synopsys-plugins.update none - -# initialize submodules and get the hashes -git submodule update --init -status=$(git submodule status) - -all_names=() - - -search_submodule() { - echo "Running check on submodule $submodule in $dir" - hash=$(echo "$status" | grep "$dir.*$submodule " | awk '{print$1}' | grep -o "[[:alnum:]]*") - for branch in "${branches[@]}" - do - echo "Searching for $hash in origin/$branch of $submodule" - (git -C $dir/$submodule branch -r --contains "$hash" | grep "origin/$branch") && true # needs init'ed submodules - if [ $? -eq 0 ] - then - all_names+=("$dir/$submodule $hash 0") - return - fi - done - all_names+=("$dir/$submodule $hash 1") - return -} - -search () { - for submodule in "${submodules[@]}" - do - search_submodule - done -} - -submodules=("cva6" "boom" "ibex" "gemmini" "hwacha" "icenet" "nvdla" "rocket-chip" "sha3" "sifive-blocks" "sifive-cache" "testchipip" "riscv-sodor") -dir="generators" -if [ "$CIRCLE_BRANCH" == "master" ] || [ "$CIRCLE_BRANCH" == "dev" ] -then - branches=("master" "main") -else - branches=("master" "main" "dev") -fi -search - -submodules=("riscv-gnu-toolchain" "riscv-isa-sim" "riscv-pk" "riscv-tests") -dir="toolchains/esp-tools" -branches=("master") -search - - -submodules=("riscv-gnu-toolchain" "riscv-isa-sim" "riscv-pk" "riscv-tests") -dir="toolchains/riscv-tools" -branches=("master") -search - -# riscv-openocd doesn't use its master branch -submodules=("riscv-openocd") -dir="toolchains/riscv-tools" -branches=("riscv") -search - -submodules=("qemu" "libgloss") -dir="toolchains" -branches=("master") -search - -submodules=("coremark" "firemarshal" "nvdla-workload" "spec2017") -dir="software" -if [ "$CIRCLE_BRANCH" == "master" ] || [ "$CIRCLE_BRANCH" == "dev" ] -then - branches=("master") -else - branches=("master" "dev") -fi -search - -submodules=("DRAMSim2" "axe" "barstools" "chisel-testers" "dsptools" "rocket-dsp-utils" "firrtl-interpreter" "torture" "treadle") -dir="tools" -if [ "$CIRCLE_BRANCH" == "master" ] || [ "$CIRCLE_BRANCH" == "dev" ] -then - branches=("master") -else - branches=("master" "dev") -fi -search - -submodules=("dromajo-src") -dir="tools/dromajo" -branches=("master") -search - -submodules=("firesim") -dir="sims" -if [ "$CIRCLE_BRANCH" == "master" ] || [ "$CIRCLE_BRANCH" == "dev" ] -then - branches=("master") -else - branches=("master" "dev") -fi -search - -submodules=("hammer") -dir="vlsi" -branches=("master") -search - -submodules=("fpga-shells") -dir="fpga" -branches=("master") -search - -# turn off verbose printing to make this easier to read -set +x - -# print 0's -for str in "${all_names[@]}"; -do - if [ 0 = $(echo "$str" | awk '{print$3}') ]; then - echo "$str" - fi -done - -echo "" - -# check if there was a non-zero return code and print 1's -EXIT=0 -for str in "${all_names[@]}"; -do - if [ ! 0 = $(echo "$str" | awk '{print$3}') ]; then - echo "$str" - EXIT=1 - fi -done - -echo "Done checking all submodules" -exit $EXIT diff --git a/.circleci/clean-old-files.sh b/.circleci/clean-old-files.sh deleted file mode 100755 index 5824c4b7..00000000 --- a/.circleci/clean-old-files.sh +++ /dev/null @@ -1,29 +0,0 @@ -#!/bin/bash - -# clean directories that are older than 14 days -# argument is used as the directory to look in - -age () { - local AGE_SEC - local CUR_SEC - local DIFF_SEC - local SEC_PER_DAY - - SEC_PER_DAY=86400 - - CUR_SEC=$(date +%s) - AGE_SEC=$(stat -c %Y -- "$1") - DIFF_SEC=$(expr $CUR_SEC - $AGE_SEC) - - echo $(expr $DIFF_SEC / $SEC_PER_DAY) -} - -for d in $1/*/ ; do - DIR_AGE="$(age $d)" - if [ $DIR_AGE -ge 14 ]; then - echo "Deleting $d since is it $DIR_AGE old" - rm -rf $d - else - echo "Keep $d since it is $DIR_AGE old" - fi -done diff --git a/.circleci/config.yml b/.circleci/config.yml deleted file mode 100644 index ec59d8ca..00000000 --- a/.circleci/config.yml +++ /dev/null @@ -1,534 +0,0 @@ -# CircleCI Configuration File - -# version of circleci -version: 2.1 - -parameters: - tools-cache-version: - type: string - default: "v13" - -# default execution env.s -executors: - main-env: - docker: - - image: ucbbar/chipyard-ci-image:554b436 - environment: - JVM_OPTS: -Xmx3200m # Customize the JVM maximum heap limit - -# re-usable commands -commands: - toolchain-build: - description: "Build a toolchain" - parameters: - tools-version: - type: string - default: "riscv-tools" - steps: - - checkout - - run: - name: Create hash of toolchains - command: | - .circleci/create-hash.sh - - restore_cache: - keys: - - << parameters.tools-version >>-installed-<< pipeline.parameters.tools-cache-version >>-{{ checksum "../<< parameters.tools-version >>.hash" }} - - run: - name: Building << parameters.tools-version >> - command: | - .circleci/build-toolchains.sh << parameters.tools-version >> - no_output_timeout: 120m - - save_cache: - key: << parameters.tools-version >>-installed-<< pipeline.parameters.tools-cache-version >>-{{ checksum "../<< parameters.tools-version >>.hash" }} - paths: - - "/root/<< parameters.tools-version >>-install" - - ssh-checkout: - description: "Add SSH key and checkout code" - steps: - - add_ssh_keys: - fingerprints: - - "3e:c3:02:5b:ed:64:8c:b7:b0:04:43:bc:83:43:73:1e" - - "32:d6:89:d2:97:fa:db:de:a8:2d:2a:f2:70:dd:80:89" - - checkout - - setup-tools: - description: "Get toolchain" - parameters: - tools-version: - type: string - default: "riscv-tools" - steps: - - ssh-checkout - - run: - name: Create hash of toolchains - command: | - .circleci/create-hash.sh - - restore_cache: - keys: - - << parameters.tools-version >>-installed-<< pipeline.parameters.tools-cache-version >>-{{ checksum "../<< parameters.tools-version >>.hash" }} - - prepare-rtl: - description: "Run the prepare step of RTL" - parameters: - tools-version: - type: string - default: "riscv-tools" - group-key: - type: string - timeout: - type: string - default: "120m" - build-script: - type: string - default: "do-rtl-build.sh" - build-type: - type: string - default: "sim" - steps: - - setup-tools: - tools-version: "<< parameters.tools-version >>" - - run: - name: Building << parameters.group-key >> subproject using Verilator - command: .circleci/<< parameters.build-script >> << parameters.group-key >> << parameters.build-type >> - no_output_timeout: << parameters.timeout >> - - save_cache: - key: << parameters.group-key >>-{{ .Branch }}-{{ .Revision }} - paths: - - "/root/project" - - run-tests: - description: "Run a set of tests" - parameters: - tools-version: - type: string - default: "riscv-tools" - group-key: - type: string - project-key: - type: string - run-script: - type: string - default: "run-tests.sh" - timeout: - type: string - default: "25m" - steps: - - setup-tools: - tools-version: "<< parameters.tools-version >>" - - restore_cache: - keys: - - << parameters.group-key >>-{{ .Branch }}-{{ .Revision }} - - run: - name: Run << parameters.project-key >> subproject tests - command: .circleci/<< parameters.run-script >> << parameters.project-key >> - no_output_timeout: << parameters.timeout >> - -# set of jobs to run -jobs: - commit-on-master-check: - executor: main-env - steps: - - checkout - - run: - name: Check commits of each submodule - command: | - .circleci/check-commit.sh - tutorial-setup-check: - executor: main-env - steps: - - checkout - - run: - name: Check that the tutorial-setup patches apply - command: | - scripts/tutorial-setup.sh - documentation-check: - executor: main-env - steps: - - checkout - - run: - name: Check that documentation builds with no warnings/errors - command: | - sudo apt-get update -y - sudo apt-get install -y python3-pip - sudo pip3 install -r docs/requirements.txt - make -C docs html - - install-riscv-toolchain: - executor: main-env - steps: - - toolchain-build: - tools-version: "riscv-tools" - install-esp-toolchain: - executor: main-env - steps: - - toolchain-build: - tools-version: "esp-tools" - install-verilator: - executor: main-env - steps: - - ssh-checkout - - run: - name: Install Verilator to remote - command: | - .circleci/install-verilator.sh - build-extra-tests: - executor: main-env - steps: - - ssh-checkout - - run: - name: Create hash of toolchains - command: | - .circleci/create-hash.sh - - restore_cache: - keys: - - riscv-tools-installed-<< pipeline.parameters.tools-cache-version >>-{{ checksum "../riscv-tools.hash" }} - - run: - name: Build extra tests - command: .circleci/build-extra-tests.sh - no_output_timeout: 120m - - save_cache: - key: extra-tests-{{ .Branch }}-{{ .Revision }} - paths: - - "/root/project/tests" - - prepare-chipyard-cores: - executor: main-env - steps: - - prepare-rtl: - group-key: "group-cores" - prepare-chipyard-peripherals: - executor: main-env - steps: - - prepare-rtl: - group-key: "group-peripherals" - prepare-chipyard-accels: - executor: main-env - steps: - - prepare-rtl: - tools-version: "esp-tools" - group-key: "group-accels" - prepare-chipyard-tracegen: - executor: main-env - steps: - - prepare-rtl: - group-key: "group-tracegen" - prepare-chipyard-other: - executor: main-env - steps: - - prepare-rtl: - group-key: "group-other" - - chipyard-rocket-run-tests: - executor: main-env - steps: - - run-tests: - group-key: "group-cores" - project-key: "chipyard-rocket" - chipyard-hetero-run-tests: - executor: main-env - steps: - - run-tests: - group-key: "group-cores" - project-key: "chipyard-hetero" - timeout: "20m" - chipyard-boom-run-tests: - executor: main-env - steps: - - run-tests: - group-key: "group-cores" - project-key: "chipyard-boom" - chipyard-cva6-run-tests: - executor: main-env - steps: - - run-tests: - group-key: "group-cores" - project-key: "chipyard-cva6" - timeout: "30m" - chipyard-sodor-run-tests: - executor: main-env - steps: - - run-tests: - group-key: "group-cores" - project-key: "chipyard-sodor" - timeout: "30m" - chipyard-ibex-run-tests: - executor: main-env - steps: - - run-tests: - group-key: "group-cores" - project-key: "chipyard-ibex" - timeout: "30m" - chipyard-multiclock-rocket-run-tests: - executor: main-env - steps: - - run-tests: - group-key: "group-cores" - project-key: "chipyard-multiclock-rocket" - chipyard-dmirocket-run-tests: - executor: main-env - steps: - - run-tests: - group-key: "group-peripherals" - project-key: "chipyard-dmirocket" - chipyard-spiflashwrite-run-tests: - executor: main-env - steps: - - run-tests: - group-key: "group-peripherals" - project-key: "chipyard-spiflashwrite" - chipyard-spiflashread-run-tests: - executor: main-env - steps: - - run-tests: - group-key: "group-peripherals" - project-key: "chipyard-spiflashread" - chipyard-lbwif-run-tests: - executor: main-env - steps: - - run-tests: - group-key: "group-peripherals" - project-key: "chipyard-lbwif" - - chipyard-sha3-run-tests: - executor: main-env - steps: - - run-tests: - tools-version: "esp-tools" - group-key: "group-accels" - project-key: "chipyard-sha3" - chipyard-streaming-fir-run-tests: - executor: main-env - steps: - - run-tests: - group-key: "group-accels" - project-key: "chipyard-streaming-fir" - chipyard-streaming-passthrough-run-tests: - executor: main-env - steps: - - run-tests: - group-key: "group-accels" - project-key: "chipyard-streaming-passthrough" - chipyard-hwacha-run-tests: - executor: main-env - steps: - - run-tests: - tools-version: "esp-tools" - group-key: "group-accels" - project-key: "chipyard-hwacha" - timeout: "60m" - chipyard-gemmini-run-tests: - executor: main-env - steps: - - run-tests: - tools-version: "esp-tools" - group-key: "group-accels" - project-key: "chipyard-gemmini" - chipyard-nvdla-run-tests: - executor: main-env - steps: - - run-tests: - group-key: "group-accels" - project-key: "chipyard-nvdla" - tracegen-run-tests: - executor: main-env - steps: - - run-tests: - group-key: "group-tracegen" - project-key: "tracegen" - tracegen-boom-run-tests: - executor: main-env - steps: - - run-tests: - group-key: "group-tracegen" - project-key: "tracegen-boom" - icenet-run-tests: - executor: main-env - steps: - - run-tests: - group-key: "group-other" - project-key: "icenet" - timeout: "30m" - testchipip-run-tests: - executor: main-env - steps: - - run-tests: - group-key: "group-other" - project-key: "testchipip" - timeout: "30m" - firesim-run-tests: - executor: main-env - steps: - - run-tests: - group-key: "extra-tests" - project-key: "firesim" - run-script: "run-firesim-scala-tests.sh" - timeout: "20m" - fireboom-run-tests: - executor: main-env - steps: - - run-tests: - group-key: "extra-tests" - project-key: "fireboom" - run-script: "run-firesim-scala-tests.sh" - timeout: "45m" - firesim-multiclock-run-tests: - executor: main-env - steps: - - run-tests: - group-key: "extra-tests" - project-key: "firesim-multiclock" - run-script: "run-firesim-scala-tests.sh" - timeout: "20m" - prepare-chipyard-fpga: - executor: main-env - steps: - - prepare-rtl: - group-key: "group-fpga" - build-type: "fpga" - -# Order and dependencies of jobs to run -workflows: - version: 2 - submodules-on-master: - jobs: - # Check to make sure submodule commits are on master branches - - commit-on-master-check - triggers: - - schedule: - cron: "0 0 * * *" - filters: - branches: - only: - - master - - build-and-test-chipyard-integration: - jobs: - # Make the toolchains - - install-riscv-toolchain - - - install-esp-toolchain - - - install-verilator - - - commit-on-master-check - - # Attempt to apply the tutorial patches - - tutorial-setup-check - - # Check that documentation builds - - documentation-check - - # Build extra tests - - build-extra-tests: - requires: - - install-riscv-toolchain - - # Prepare the verilator builds - - prepare-chipyard-cores: - requires: - - install-riscv-toolchain - - install-verilator - - prepare-chipyard-peripherals: - requires: - - install-riscv-toolchain - - install-verilator - - prepare-chipyard-accels: - requires: - - install-esp-toolchain - - install-verilator - - prepare-chipyard-tracegen: - requires: - - install-riscv-toolchain - - install-verilator - - prepare-chipyard-other: - requires: - - install-riscv-toolchain - - install-verilator - - # Run the example tests - - chipyard-rocket-run-tests: - requires: - - prepare-chipyard-cores - - chipyard-hetero-run-tests: - requires: - - prepare-chipyard-cores - - chipyard-boom-run-tests: - requires: - - prepare-chipyard-cores - - chipyard-cva6-run-tests: - requires: - - prepare-chipyard-cores - - chipyard-sodor-run-tests: - requires: - - prepare-chipyard-cores - - chipyard-ibex-run-tests: - requires: - - prepare-chipyard-cores - - chipyard-dmirocket-run-tests: - requires: - - prepare-chipyard-peripherals - - chipyard-spiflashwrite-run-tests: - requires: - - prepare-chipyard-peripherals - - chipyard-spiflashread-run-tests: - requires: - - prepare-chipyard-peripherals - - chipyard-lbwif-run-tests: - requires: - - prepare-chipyard-peripherals - - - chipyard-sha3-run-tests: - requires: - - prepare-chipyard-accels - - chipyard-streaming-fir-run-tests: - requires: - - prepare-chipyard-accels - - chipyard-streaming-passthrough-run-tests: - requires: - - prepare-chipyard-accels - - chipyard-hwacha-run-tests: - requires: - - prepare-chipyard-accels - - chipyard-gemmini-run-tests: - requires: - - prepare-chipyard-accels - - chipyard-nvdla-run-tests: - requires: - - prepare-chipyard-accels - - - tracegen-run-tests: - requires: - - prepare-chipyard-tracegen - - tracegen-boom-run-tests: - requires: - - prepare-chipyard-tracegen - - - icenet-run-tests: - requires: - - prepare-chipyard-other - - testchipip-run-tests: - requires: - - prepare-chipyard-other - - # Run the firesim tests - - firesim-run-tests: - requires: - - install-riscv-toolchain - - install-verilator - - build-extra-tests - - firesim-multiclock-run-tests: - requires: - - install-riscv-toolchain - - install-verilator - - build-extra-tests - - fireboom-run-tests: - requires: - - install-riscv-toolchain - - install-verilator - - build-extra-tests - - # Prepare the fpga builds (just Verilog) - - prepare-chipyard-fpga: - requires: - - install-riscv-toolchain diff --git a/.circleci/create-hash.sh b/.circleci/create-hash.sh deleted file mode 100755 index 7a8915a5..00000000 --- a/.circleci/create-hash.sh +++ /dev/null @@ -1,23 +0,0 @@ -#!/bin/bash - -# get the hash of riscv-tools - -# turn echo on and error on earliest command -set -ex -set -o pipefail - -# get shared variables -SCRIPT_DIR="$( cd "$( dirname "$0" )" && pwd )" -source $SCRIPT_DIR/defaults.sh - -# enter bhd repo -cd $LOCAL_CHIPYARD_DIR - -# Use normalized output of git-submodule status as hashfile -for tools in 'riscv-tools' 'esp-tools' ; do - git submodule status "toolchains/${tools}" 'toolchains/libgloss' 'toolchains/qemu' | - while read -r line ; do - echo "${line#[!0-9a-f]}" - done > "${HOME}/${tools}.hash" -done -echo "Hashfile for riscv-tools and esp-tools created in $HOME" diff --git a/.circleci/defaults.sh b/.circleci/defaults.sh deleted file mode 100755 index 2ff81ef1..00000000 --- a/.circleci/defaults.sh +++ /dev/null @@ -1,93 +0,0 @@ -#!/bin/bash - -copy () { - rsync -avzp -e 'ssh' --exclude '.git' $1 $2 -} - -run () { - ssh -o "StrictHostKeyChecking no" -t $SERVER $@ -} - -run_script () { - ssh -o "StrictHostKeyChecking no" -t $SERVER 'bash -s' < $1 "$2" -} - -clean () { - # remove remote work dir - run "rm -rf $REMOTE_WORK_DIR" -} - -# make parallelism -CI_MAKE_NPROC=8 -# chosen based on a 24c system shared with 1 other project -REMOTE_MAKE_NPROC=4 - -# verilator version -VERILATOR_VERSION=v4.034 - -# remote variables -REMOTE_PREFIX=$CI_DIR/$CIRCLE_PROJECT_REPONAME-$CIRCLE_BRANCH -REMOTE_WORK_DIR=$REMOTE_PREFIX-$CIRCLE_SHA1-$CIRCLE_JOB -REMOTE_RISCV_DIR=$REMOTE_WORK_DIR/riscv-tools-install -REMOTE_ESP_DIR=$REMOTE_WORK_DIR/esp-tools-install -REMOTE_CHIPYARD_DIR=$REMOTE_WORK_DIR/chipyard -REMOTE_SIM_DIR=$REMOTE_CHIPYARD_DIR/sims/verilator -REMOTE_FIRESIM_DIR=$REMOTE_CHIPYARD_DIR/sims/firesim/sim -REMOTE_FPGA_DIR=$REMOTE_CHIPYARD_DIR/fpga -REMOTE_JAVA_OPTS="-Xmx10G -Xss8M" -# Disable the supershell to greatly improve the readability of SBT output when captured by Circle CI -REMOTE_SBT_OPTS="-Dsbt.ivy.home=$REMOTE_WORK_DIR/.ivy2 -Dsbt.supershell=false -Dsbt.global.base=$REMOTE_WORK_DIR/.sbt -Dsbt.boot.directory=$REMOTE_WORK_DIR/.sbt/boot" -REMOTE_VERILATOR_DIR=$REMOTE_PREFIX-$CIRCLE_SHA1-verilator-install - -# local variables (aka within the docker container) -LOCAL_CHECKOUT_DIR=$HOME/project -LOCAL_RISCV_DIR=$HOME/riscv-tools-install -LOCAL_ESP_DIR=$HOME/esp-tools-install -LOCAL_CHIPYARD_DIR=$LOCAL_CHECKOUT_DIR -LOCAL_SIM_DIR=$LOCAL_CHIPYARD_DIR/sims/verilator -LOCAL_FIRESIM_DIR=$LOCAL_CHIPYARD_DIR/sims/firesim/sim - -# key value store to get the build groups -declare -A grouping -grouping["group-cores"]="chipyard-cva6 chipyard-ibex chipyard-rocket chipyard-hetero chipyard-boom chipyard-sodor chipyard-digitaltop chipyard-multiclock-rocket chipyard-nomem-scratchpad chipyard-fftgenerator" -grouping["group-peripherals"]="chipyard-dmirocket chipyard-blkdev chipyard-spiflashread chipyard-spiflashwrite chipyard-mmios chipyard-lbwif" -grouping["group-accels"]="chipyard-nvdla chipyard-sha3 chipyard-hwacha chipyard-gemmini chipyard-streaming-fir chipyard-streaming-passthrough" -grouping["group-tracegen"]="tracegen tracegen-boom" -grouping["group-other"]="icenet testchipip" -grouping["group-fpga"]="arty vcu118" - -# key value store to get the build strings -declare -A mapping -mapping["chipyard-rocket"]="" -mapping["chipyard-dmirocket"]=" CONFIG=dmiRocketConfig" -mapping["chipyard-lbwif"]=" CONFIG=LBWIFRocketConfig" -mapping["chipyard-sha3"]=" CONFIG=Sha3RocketConfig" -mapping["chipyard-digitaltop"]=" TOP=DigitalTop" -mapping["chipyard-streaming-fir"]=" CONFIG=StreamingFIRRocketConfig" -mapping["chipyard-streaming-passthrough"]=" CONFIG=StreamingPassthroughRocketConfig" -mapping["chipyard-hetero"]=" CONFIG=LargeBoomAndRocketConfig" -mapping["chipyard-boom"]=" CONFIG=SmallBoomConfig" -mapping["chipyard-blkdev"]=" CONFIG=SimBlockDeviceRocketConfig" -mapping["chipyard-hwacha"]=" CONFIG=HwachaRocketConfig" -mapping["chipyard-gemmini"]=" CONFIG=GemminiRocketConfig" -mapping["chipyard-cva6"]=" CONFIG=CVA6Config" -mapping["chipyard-ibex"]=" CONFIG=IbexConfig" -mapping["chipyard-spiflashread"]=" CONFIG=LargeSPIFlashROMRocketConfig" -mapping["chipyard-spiflashwrite"]=" CONFIG=SmallSPIFlashRocketConfig" -mapping["chipyard-mmios"]=" CONFIG=MMIORocketConfig verilog" -mapping["tracegen"]=" CONFIG=NonBlockingTraceGenL2Config" -mapping["tracegen-boom"]=" CONFIG=BoomTraceGenConfig" -mapping["chipyard-nvdla"]=" CONFIG=SmallNVDLARocketConfig" -mapping["chipyard-sodor"]=" CONFIG=Sodor5StageConfig" -mapping["chipyard-multiclock-rocket"]=" CONFIG=MulticlockRocketConfig" -mapping["chipyard-nomem-scratchpad"]=" CONFIG=MMIOScratchpadOnlyRocketConfig" -mapping["chipyard-fftgenerator"]=" CONFIG=FFTRocketConfig" - -mapping["firesim"]="SCALA_TEST=firesim.firesim.RocketNICF1Tests" -mapping["firesim-multiclock"]="SCALA_TEST=firesim.firesim.RocketMulticlockF1Tests" -mapping["fireboom"]="SCALA_TEST=firesim.firesim.BoomF1Tests" -mapping["icenet"]="SUB_PROJECT=icenet" -mapping["testchipip"]="SUB_PROJECT=testchipip" - -mapping["arty"]="SUB_PROJECT=arty verilog" -mapping["vcu118"]="SUB_PROJECT=vcu118 verilog" diff --git a/.circleci/do-rtl-build.sh b/.circleci/do-rtl-build.sh deleted file mode 100755 index 3a537ea3..00000000 --- a/.circleci/do-rtl-build.sh +++ /dev/null @@ -1,92 +0,0 @@ -#!/bin/bash - -# create the different verilator builds -# usage: -# do-rtl-build.sh sim -# run rtl build for simulations and copy back results -# do-rtl-build.sh fpga -# run rtl build for fpga and don't copy back results - -# turn echo on and error on earliest command -set -ex - -# get shared variables -SCRIPT_DIR="$( cd "$( dirname "$0" )" && pwd )" -source $SCRIPT_DIR/defaults.sh - -# call clean on exit -trap clean EXIT - -cd $LOCAL_CHIPYARD_DIR -./scripts/init-submodules-no-riscv-tools.sh -./scripts/init-fpga.sh - -# set stricthostkeychecking to no (must happen before rsync) -run "echo \"Ping $SERVER\"" - -clean - -# copy over riscv/esp-tools, and chipyard to remote -run "mkdir -p $REMOTE_CHIPYARD_DIR" -copy $LOCAL_CHIPYARD_DIR/ $SERVER:$REMOTE_CHIPYARD_DIR - -run "cp -r ~/.ivy2 $REMOTE_WORK_DIR" -run "cp -r ~/.sbt $REMOTE_WORK_DIR" - -TOOLS_DIR=$REMOTE_RISCV_DIR -LD_LIB_DIR=$REMOTE_RISCV_DIR/lib - -if [ $1 = "group-accels" ]; then - export RISCV=$LOCAL_ESP_DIR - export LD_LIBRARY_PATH=$LOCAL_ESP_DIR/lib - export PATH=$RISCV/bin:$PATH - GEMMINI_SOFTWARE_DIR=$LOCAL_SIM_DIR/../../generators/gemmini/software/gemmini-rocc-tests - cd $LOCAL_SIM_DIR/../../generators/gemmini/software - git submodule update --init --recursive gemmini-rocc-tests - cd gemmini-rocc-tests - ./build.sh - - TOOLS_DIR=$REMOTE_ESP_DIR - LD_LIB_DIR=$REMOTE_ESP_DIR/lib - run "mkdir -p $REMOTE_ESP_DIR" - copy $LOCAL_ESP_DIR/ $SERVER:$REMOTE_ESP_DIR -else - run "mkdir -p $REMOTE_RISCV_DIR" - copy $LOCAL_RISCV_DIR/ $SERVER:$REMOTE_RISCV_DIR -fi - -# choose what make dir to use -case $2 in - "sim") - REMOTE_MAKE_DIR=$REMOTE_SIM_DIR - ;; - "fpga") - REMOTE_MAKE_DIR=$REMOTE_FPGA_DIR - ;; -esac - -# enter the verilator directory and build the specific config on remote server -run "export RISCV=\"$TOOLS_DIR\"; \ - make -C $REMOTE_MAKE_DIR clean;" - -read -a keys <<< ${grouping[$1]} - -# need to set the PATH to use the new verilator (with the new verilator root) -for key in "${keys[@]}" -do - run "export RISCV=\"$TOOLS_DIR\"; \ - export LD_LIBRARY_PATH=\"$LD_LIB_DIR\"; \ - export PATH=\"$REMOTE_VERILATOR_DIR/bin:\$PATH\"; \ - export VERILATOR_ROOT=\"$REMOTE_VERILATOR_DIR\"; \ - export COURSIER_CACHE=\"$REMOTE_WORK_DIR/.coursier-cache\"; \ - make -j$REMOTE_MAKE_NPROC -C $REMOTE_MAKE_DIR FIRRTL_LOGLEVEL=info JAVA_TOOL_OPTIONS=\"$REMOTE_JAVA_OPTS\" SBT_OPTS=\"$REMOTE_SBT_OPTS\" ${mapping[$key]}" -done - -run "rm -rf $REMOTE_CHIPYARD_DIR/project" - -# choose to copy back results -if [ $2 = "sim" ]; then - # copy back the final build - mkdir -p $LOCAL_CHIPYARD_DIR - copy $SERVER:$REMOTE_CHIPYARD_DIR/ $LOCAL_CHIPYARD_DIR -fi diff --git a/.circleci/install-verilator.sh b/.circleci/install-verilator.sh deleted file mode 100755 index 2170768a..00000000 --- a/.circleci/install-verilator.sh +++ /dev/null @@ -1,25 +0,0 @@ -#!/bin/bash - -# move verilator to the remote server - -# turn echo on and error on earliest command -set -ex - -# get shared variables -SCRIPT_DIR="$( cd "$( dirname "$0" )" && pwd )" -source $SCRIPT_DIR/defaults.sh - -# clean older directories (delete prior directories related to this branch also) -run_script $LOCAL_CHIPYARD_DIR/.circleci/clean-old-files.sh $CI_DIR -run "rm -rf $REMOTE_PREFIX*" - -# set stricthostkeychecking to no (must happen before rsync) -run "echo \"Ping $SERVER\"" - -run "git clone http://git.veripool.org/git/verilator $REMOTE_VERILATOR_DIR; \ - cd $REMOTE_VERILATOR_DIR; \ - git checkout $VERILATOR_VERSION; \ - autoconf; \ - export VERILATOR_ROOT=$REMOTE_VERILATOR_DIR; \ - ./configure; \ - make -j$REMOTE_MAKE_NPROC;" diff --git a/.circleci/run-firesim-scala-tests.sh b/.circleci/run-firesim-scala-tests.sh deleted file mode 100755 index 022894f2..00000000 --- a/.circleci/run-firesim-scala-tests.sh +++ /dev/null @@ -1,59 +0,0 @@ -#!/bin/bash - -# create the different verilator builds -# argument is the make command string - -# turn echo on and error on earliest command -set -ex - -# get shared variables -SCRIPT_DIR="$( cd "$( dirname "$0" )" && pwd )" -source $SCRIPT_DIR/defaults.sh - -# call clean on exit -trap clean EXIT - -# Directory locations for handling firesim-local installations of libelf/libdwarf -# This would generally be handled by build-setup.sh/firesim-setup.sh -firesim_sysroot=lib-install -local_firesim_sysroot=$LOCAL_FIRESIM_DIR/$firesim_sysroot -remote_firesim_sysroot=$REMOTE_FIRESIM_DIR/$firesim_sysroot - -cd $LOCAL_CHIPYARD_DIR -./scripts/init-submodules-no-riscv-tools.sh -cd $LOCAL_CHIPYARD_DIR/sims/firesim/sim/firesim-lib/src/main/cc/lib -git submodule update --init elfutils libdwarf -cd $LOCAL_CHIPYARD_DIR/sims/firesim -mkdir -p $local_firesim_sysroot -./scripts/build-libelf.sh $local_firesim_sysroot -./scripts/build-libdwarf.sh $local_firesim_sysroot -cd $LOCAL_CHIPYARD_DIR - -make -C $LOCAL_CHIPYARD_DIR/tools/dromajo/dromajo-src/src - -# set stricthostkeychecking to no (must happen before rsync) -run "echo \"Ping $SERVER\"" - -clean - -# copy over riscv/esp-tools, and chipyard to remote -run "mkdir -p $REMOTE_CHIPYARD_DIR" -run "mkdir -p $REMOTE_RISCV_DIR" -copy $LOCAL_CHIPYARD_DIR/ $SERVER:$REMOTE_CHIPYARD_DIR -copy $LOCAL_RISCV_DIR/ $SERVER:$REMOTE_RISCV_DIR - -run "cp -r ~/.ivy2 $REMOTE_WORK_DIR" -run "cp -r ~/.sbt $REMOTE_WORK_DIR" - -TOOLS_DIR=$REMOTE_RISCV_DIR - -LD_LIB_DIR=$remote_firesim_sysroot/lib:$REMOTE_RISCV_DIR/lib - -# Run Firesim Scala Tests -run "export RISCV=\"$TOOLS_DIR\"; \ - export LD_LIBRARY_PATH=\"$LD_LIB_DIR\"; \ - export FIRESIM_ENV_SOURCED=1; \ - export PATH=\"$REMOTE_VERILATOR_DIR/bin:\$PATH\"; \ - export VERILATOR_ROOT=\"$REMOTE_VERILATOR_DIR\"; \ - export COURSIER_CACHE=\"$REMOTE_WORK_DIR/.coursier-cache\"; \ - make -C $REMOTE_FIRESIM_DIR JAVA_TOOL_OPTIONS=\"$REMOTE_JAVA_OPTS\" SBT_OPTS=\"$REMOTE_SBT_OPTS\" testOnly ${mapping[$1]}" diff --git a/.circleci/run-tests.sh b/.circleci/run-tests.sh deleted file mode 100755 index 461843fd..00000000 --- a/.circleci/run-tests.sh +++ /dev/null @@ -1,120 +0,0 @@ -#!/bin/bash - -# run the different tests - -# turn echo on and error on earliest command -set -ex - -# get remote exec variables -SCRIPT_DIR="$( cd "$( dirname "$0" )" && pwd )" -source $SCRIPT_DIR/defaults.sh - -run_bmark () { - make run-bmark-tests-fast -j$CI_MAKE_NPROC -C $LOCAL_SIM_DIR $@ -} - -run_asm () { - make run-asm-tests-fast -j$CI_MAKE_NPROC -C $LOCAL_SIM_DIR $@ -} - -run_both () { - run_bmark $@ - run_asm $@ -} - -run_tracegen () { - make tracegen -C $LOCAL_SIM_DIR $@ -} - -# TODO BUG: the run-binary command forces a rebuild of the simulator in CI -# instead, directly run the simulator binary -case $1 in - chipyard-rocket) - run_bmark ${mapping[$1]} - ;; - chipyard-dmirocket) - run_bmark ${mapping[$1]} - ;; - chipyard-lbwif) - run_bmark ${mapping[$1]} - ;; - chipyard-boom) - run_bmark ${mapping[$1]} - ;; - chipyard-hetero) - run_bmark ${mapping[$1]} - ;; - rocketchip) - run_bmark ${mapping[$1]} - ;; - chipyard-hwacha) - export RISCV=$LOCAL_ESP_DIR - export LD_LIBRARY_PATH=$LOCAL_ESP_DIR/lib - export PATH=$RISCV/bin:$PATH - make run-rv64uv-p-asm-tests -j$CI_MAKE_NPROC -C $LOCAL_SIM_DIR ${mapping[$1]} - ;; - chipyard-gemmini) - export RISCV=$LOCAL_ESP_DIR - export LD_LIBRARY_PATH=$LOCAL_ESP_DIR/lib - export PATH=$RISCV/bin:$PATH - GEMMINI_SOFTWARE_DIR=$LOCAL_SIM_DIR/../../generators/gemmini/software/gemmini-rocc-tests - rm -rf $GEMMINI_SOFTWARE_DIR/riscv-tests - cd $LOCAL_SIM_DIR - $LOCAL_SIM_DIR/simulator-chipyard-GemminiRocketConfig $GEMMINI_SOFTWARE_DIR/build/bareMetalC/aligned-baremetal - $LOCAL_SIM_DIR/simulator-chipyard-GemminiRocketConfig $GEMMINI_SOFTWARE_DIR/build/bareMetalC/raw_hazard-baremetal - $LOCAL_SIM_DIR/simulator-chipyard-GemminiRocketConfig $GEMMINI_SOFTWARE_DIR/build/bareMetalC/mvin_mvout-baremetal - ;; - chipyard-sha3) - export RISCV=$LOCAL_ESP_DIR - export LD_LIBRARY_PATH=$LOCAL_ESP_DIR/lib - export PATH=$RISCV/bin:$PATH - (cd $LOCAL_CHIPYARD_DIR/generators/sha3/software && ./build.sh) - $LOCAL_SIM_DIR/simulator-chipyard-Sha3RocketConfig $LOCAL_CHIPYARD_DIR/generators/sha3/software/tests/bare/sha3-rocc.riscv - ;; - chipyard-streaming-passthrough) - make -C $LOCAL_CHIPYARD_DIR/tests - $LOCAL_SIM_DIR/simulator-chipyard-StreamingPassthroughRocketConfig $LOCAL_CHIPYARD_DIR/tests/streaming-passthrough.riscv - ;; - chipyard-streaming-fir) - make -C $LOCAL_CHIPYARD_DIR/tests - $LOCAL_SIM_DIR/simulator-chipyard-StreamingFIRRocketConfig $LOCAL_CHIPYARD_DIR/tests/streaming-fir.riscv - ;; - chipyard-spiflashread) - make -C $LOCAL_CHIPYARD_DIR/tests - make -C $LOCAL_SIM_DIR ${mapping[$1]} BINARY=$LOCAL_CHIPYARD_DIR/tests/spiflashread.riscv SIM_FLAGS="+spiflash0=${LOCAL_CHIPYARD_DIR}/tests/spiflash.img" run-binary - ;; - chipyard-spiflashwrite) - make -C $LOCAL_CHIPYARD_DIR/tests - make -C $LOCAL_SIM_DIR ${mapping[$1]} BINARY=$LOCAL_CHIPYARD_DIR/tests/spiflashwrite.riscv SIM_FLAGS="+spiflash0=${LOCAL_CHIPYARD_DIR}/tests/spiflash.img" run-binary - [[ "`xxd $LOCAL_CHIPYARD_DIR/tests/spiflash.img | grep 1337\ 00ff\ aa55\ face | wc -l`" == "6" ]] || false - ;; - tracegen) - run_tracegen ${mapping[$1]} - ;; - tracegen-boom) - run_tracegen ${mapping[$1]} - ;; - chipyard-cva6) - make run-binary-fast -C $LOCAL_SIM_DIR ${mapping[$1]} BINARY=$RISCV/riscv64-unknown-elf/share/riscv-tests/benchmarks/multiply.riscv - ;; - chipyard-sodor) - run_asm ${mapping[$1]} - ;; - chipyard-ibex) - run_bmark ${mapping[$1]} #TODO: Find 32-bit test - ;; - chipyard-nvdla) - make -C $LOCAL_CHIPYARD_DIR/tests - make -C $LOCAL_SIM_DIR ${mapping[$1]} BINARY=$LOCAL_CHIPYARD_DIR/tests/nvdla.riscv run-binary - ;; - icenet) - make run-binary-fast BINARY=none -C $LOCAL_SIM_DIR ${mapping[$1]} - ;; - testchipip) - make run-binary-fast BINARY=none -C $LOCAL_SIM_DIR ${mapping[$1]} - ;; - *) - echo "No set of tests for $1. Did you spell it right?" - exit 1 - ;; -esac