From d204ccd9fc1ec4fcc6de4f84c8db06e0500e55fa Mon Sep 17 00:00:00 2001 From: Abraham Gonzalez Date: Tue, 9 Mar 2021 23:56:32 -0800 Subject: [PATCH] Clean up the chip communication docs a bit more [ci skip] --- docs/Advanced-Concepts/Chip-Communication.rst | 87 ++++++++++++------- 1 file changed, 57 insertions(+), 30 deletions(-) diff --git a/docs/Advanced-Concepts/Chip-Communication.rst b/docs/Advanced-Concepts/Chip-Communication.rst index dc4c95cf..5c18783d 100644 --- a/docs/Advanced-Concepts/Chip-Communication.rst +++ b/docs/Advanced-Concepts/Chip-Communication.rst @@ -78,7 +78,6 @@ command into a TileLink request. This conversion is done by the DTM named ``Debu When the DTM receives the program to load, it starts to write the binary byte-wise into memory. This is considerably slower than the TSI protocol communication pipeline (i.e. ``SimSerial``/``SerialAdapter``/TileLink) which directly writes the program binary to memory. -Thus, Chipyard removes the DTM by default in favor of the TSI protocol for DUT communication. Starting the TSI or DMI Simulation ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -117,26 +116,48 @@ Then you can run simulations with the new DMI-enabled top-level and test-harness Using the JTAG Interface ------------------------ -The main way to use JTAG with a Rocket Chip based system is to instantiate the Debug Transfer Module (DTM) -and configure it to use a JTAG interface. -The default Chipyard designs instantiate the DTM and configure it to use JTAG. -You may attach OpenOCD and GDB to any of the default JTAG-enabled designs. +Another way to interface with the DUT is to use JTAG. +Similar to the :ref:`Advanced-Concepts/Chip-Communication:Using the Debug Module interface (DMI)` section, in order to use the JTAG protocol, +the DUT needs to contain a Debug Transfer Module (DTM) configured to use JTAG instead of DMI. +Once the JTAG port is exposed, the host can communicate over JTAG to the DUT through a simulation stub +called ``SimJTAG`` (C++ class) that resides in a ``SimJTAG`` Verilog module (both reside in the ``generators/rocket-chip`` project). +This simulation stub creates a socket that OpenOCD and GDB can connect to when the simulation is running. +The default Chipyard designs instantiate the DTM configured to use JTAG (i.e. ``RocketConfig``). + +.. note:: + As mentioned, default Chipyard designs are enabled with JTAG. + However, they also use TSI/Serialized-TL with FESVR in case the JTAG interface isn't used. + This allows users to choose how to communicate with the DUT (use TSI or JTAG). Debugging with JTAG ~~~~~~~~~~~~~~~~~~~ -Please refer to the following resources on how to debug a Rocket Chip system with JTAG. +Roughly the steps to debug with JTAG in simulation are as follows: -* https://github.com/chipsalliance/rocket-chip#-debugging-with-gdb -* https://github.com/riscv/riscv-isa-sim#debugging-with-gdb +1. Build a Chipyard JTAG-enabled RTL design. Remember default Chipyard designs are JTAG ready. -Roughly the steps are as follows (refer to the links for details): +.. code-block:: bash -1. Build a Chipyard JTAG-enabled RTL design (i.e. ``make CONFIG=RocketConfig`` in ``sims/*``) -2. Run the simulation with remote bit-bang enabled (i.e. ``make CONFIG=RocketConfig BINARY= SIM_FLAGS="+jtag_rbb_enable=1 --rbb-port=9823" run-binary``) -3. Launch OpenOCD -4. Connect to OpenOCD via GDB -5. Done! + cd sims/verilator + # or + cd sims/vcs + + make CONFIG=RocketConfig + +2. Run the simulation with remote bit-bang enabled. Since we hope to load/run the binary using JTAG, + we can pass ``none`` as a binary (prevents FESVR from loading the program). (Adapted from: https://github.com/chipsalliance/rocket-chip#3-launch-the-emulator) + +.. code-block:: bash + + # note: this uses Chipyard make invocation to run the simulation to properly wrap the simulation args + make CONFIG=RocketConfig BINARY=none SIM_FLAGS="+jtag_rbb_enable=1 --rbb-port=9823" run-binary + +3. `Follow the instructions here to connect to the simulation using OpenOCD + GDB. `__ + +.. note:: + This section was adapted from the instruction in Rocket Chip and riscv-isa-sim. For more information refer + to that documentation: `Rocket Chip GDB Docs `__, + `riscv-isa-sim GDB Docs `__ Example Test Chip Bringup Communication --------------------------------------- @@ -144,7 +165,7 @@ Example Test Chip Bringup Communication Intro to Typical Chipyard Test Chip ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Most, if not all, Chipyard configurations are tethered using TSI (over a serial link) and have access +Most, if not all, Chipyard configurations are tethered using TSI (over a serial-link) and have access to external memory through an AXI port (backing AXI memory). The following image shows the DUT with these set of default signals: @@ -153,14 +174,14 @@ The following image shows the DUT with these set of default signals: In this setup, the serial-link is connected to the TSI/FESVR peripherals while the AXI port is connected to a simulated AXI memory. However, AXI ports tend to have many signals associated with them so instead of creating an AXI port off the DUT, -one can send the memory transactions over the bi-directional serial-link (``TLSerdesser``) so that main -interface to the DUT is the serial-link (which as comparatively less signals than an AXI port). +one can send the memory transactions over the bi-directional serial-link (``TLSerdesser``) so that the main +interface to the DUT is the serial-link (which has comparatively less signals than an AXI port). This new setup (shown below) is a typical Chipyard test chip setup: .. image:: ../_static/images/bringup-chipyard-config-communication.png -Simulation Setup -~~~~~~~~~~~~~~~~ +Simulation Setup of the Example Test Chip +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To test this type of configuration (TSI/memory transactions over the serial-link), most of the same TSI collateral would be used. @@ -171,8 +192,12 @@ serial-link. .. note:: Here the simulated AXI memory and the converters can be in a different clock domain in the test harness - than the DUT. This is done in a harness binder doing these offchip connections. See - :ref:`Advanced-Concepts/Harness-Clocks:Creating Clocks in the Test Harness` on how to generate a clock in a harness binder. + than the reference clock of the DUT. + For example, the DUT can be clocked at 3.2GHz while the simulated AXI memory can be clocked at 1GHz. + This functionality is done in the harness binder that instantiates the TSI collateral, TL-to-AXI converters, + and simulated AXI memory. + See :ref:`Advanced-Concepts/Harness-Clocks:Creating Clocks in the Test Harness` on how to generate a clock + in a harness binder. This type of simulation setup is done in the following multi-clock configuration: @@ -181,18 +206,20 @@ This type of simulation setup is done in the following multi-clock configuration :start-after: DOC include start: MulticlockAXIOverSerialConfig :end-before: DOC include end: MulticlockAXIOverSerialConfig -Real-world FPGA Setup -~~~~~~~~~~~~~~~~~~~~~ +Bringup Setup of the Example Test Chip after Tapeout +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Assuming this same chip configuration is being tested after tapeout (during bringup), -communication with the DUT is similar but slightly different. -For example, some Berkeley tapeouts have a FPGA with a RISC-V soft-core that runs FESVR. +Assuming this example test chip is taped out and now ready to be tested, we can communicate with the chip using this serial-link. +For example, some Berkeley tapeouts of Chipyard chips have an FPGA running a RISC-V soft-core that is able to speak to the DUT. This RISC-V soft-core would serve as the host of the test that will run on the DUT. -The FESVR on the soft-core sends TSI commands to the ``SerialAdapter`` which then sends the converted -TileLink transaction to the chip over the serial-link. -If the chip requests offchip memory, it can then send the transaction back over the serial-link to the -FPGA DRAM. +This is done by the RISC-V soft-core running FESVR, sending TSI commands to a ``SerialAdapter`` / ``TLSerdesser`` programmed on the FPGA. +Once the commands are converted to serialized TileLink, then they can be sent over some medium to the DUT +(like an FMC cable or a set of wires connecting FPGA outputs to the DUT board). +Similar to simulation, if the chip requests offchip memory, it can then send the transaction back over the serial-link. +Then the request can be serviced by the channel of FPGA DRAM. The following image shows this flow: .. image:: ../_static/images/chip-bringup.png +In fact, this exact type of bringup setup is what the following section discusses: +:ref:`Prototyping/VCU118:Introduction to the Bringup Platform`.