wu-arch/chipyard
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Merge pull request #1860 from ucb-bar/remove_generatemodelstagemain

Browse Source 
Remove tapeout.GenerateModelStageMain
This commit is contained in:
Jerry Zhao
2024-04-29 19:19:33 -07:00
committed by GitHub
parent 25589d611a b9339efd86
commit 0a76d61e83
14 changed files with 4 additions and 440 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
.github/scripts/remote-do-rtl-build.sh vendored
View File

@@ -53,5 +53,5 @@ do
export COURSIER_CACHE=$REMOTE_COURSIER_CACHE
export JVM_MEMORY=10G
export JAVA_TMP_DIR=$REMOTE_JAVA_TMP_DIR
make -j$REMOTE_MAKE_NPROC -C $REMOTE_MAKE_DIR FIRRTL_LOGLEVEL=info ${mapping[$key]}
make -j$REMOTE_MAKE_NPROC -C $REMOTE_MAKE_DIR ${mapping[$key]}
done

17
common.mk
View File

@@ -192,19 +192,8 @@ else
echo "$(MFC_BASE_LOWERING_OPTIONS),disallowPackedArrays" > $@
endif
$(SFC_MFC_TARGETS) &: $(TAPEOUT_CLASSPATH_TARGETS) $(FIRRTL_FILE) $(FINAL_ANNO_FILE) $(MFC_LOWERING_OPTIONS)
$(SFC_MFC_TARGETS) &: $(FIRRTL_FILE) $(FINAL_ANNO_FILE) $(MFC_LOWERING_OPTIONS)
rm -rf $(GEN_COLLATERAL_DIR)
$(call run_jar_scala_main,$(TAPEOUT_CLASSPATH),tapeout.transforms.GenerateModelStageMain,\
--no-dedup \
--output-file $(SFC_FIRRTL_BASENAME) \
--output-annotation-file $(SFC_ANNO_FILE) \
--target-dir $(GEN_COLLATERAL_DIR) \
--input-file $(FIRRTL_FILE) \
--annotation-file $(FINAL_ANNO_FILE) \
--log-level $(FIRRTL_LOGLEVEL) \
-X none \
--allow-unrecognized-annotations)
-mv $(SFC_FIRRTL_BASENAME).lo.fir $(SFC_FIRRTL_FILE)
firtool \
--format=fir \
--export-module-hierarchy \
@@ -216,10 +205,10 @@ $(SFC_MFC_TARGETS) &: $(TAPEOUT_CLASSPATH_TARGETS) $(FIRRTL_FILE) $(FINAL_ANNO_F
--lowering-options=$(shell cat $(MFC_LOWERING_OPTIONS)) \
--repl-seq-mem \
--repl-seq-mem-file=$(MFC_SMEMS_CONF) \
--annotation-file=$(SFC_ANNO_FILE) \
--annotation-file=$(FINAL_ANNO_FILE) \
--split-verilog \
-o $(GEN_COLLATERAL_DIR) \
$(SFC_FIRRTL_FILE)
$(FIRRTL_FILE)
$(SED) -i 's/.*/& /' $(MFC_SMEMS_CONF) # need trailing space for SFC macrocompiler
touch $(MFC_BB_MODS_FILELIST) # if there are no BB's then the file might not be generated, instead always generate it
# DOC include end: FirrtlCompiler

103
docs/Customization/Firrtl-Transforms.rst
View File

@@ -1,103 +0,0 @@
.. _firrtl-transforms:
Adding a Firrtl Transform
=========================
Similar to how LLVM IR passes can perform transformations and optimizations on software, FIRRTL transforms can
modify Chisel-elaborated RTL.
As mentioned in Section :ref:`Tools/FIRRTL:firrtl`, transforms are modifications that happen on the FIRRTL IR that can modify a circuit.
Transforms are a powerful tool to take in the FIRRTL IR that is emitted from Chisel and run analysis or convert the circuit into a new form.
The MLIR FIRRTL Compiler
------------------------------------------------------
In Chipyard, the LLVM-based MLIR FIRRTL compiler (CIRCT or MFC) compiles Chisel into Verilog.
For more information on MLIR FIRRTL Compiler, please visit https://mlir.llvm.org/ and https://circt.llvm.org/.
Where to add transforms
-----------------------
In Chipyard, the FIRRTL compiler is called multiple times to create a "Top" file that contains the DUT and a "Model" file containing the test harness, which instantiates the DUT.
The "Model" file does not contain the DUT's module definition or any of its submodules.
This is done by the ``tapeout`` SBT project (located in ``tools/tapeout``) which calls ``GenerateModelStageMain`` (a function that wraps the multiple FIRRTL compiler calls and extra transforms).
.. literalinclude:: ../../common.mk
:language: make
:start-after: DOC include start: FirrtlCompiler
:end-before: DOC include end: FirrtlCompiler
If you look inside of the ``tools/tapeout/src/main/scala/transforms/GenerateModelStageMain.scala`` file,
you can see that FIRRTL is invoked for "Model". Currently, the FIRRTL compiler is agnostic to the ``TOP`` and ``MODEL`` differentiation,
and the user is responsible for providing annotations that will inform the compiler where(``TOP`` vs ``MODEL``) to perform the custom FIRRTL transformations.
For more information on the Tapeout sub-project, please visit the :ref:`Tools/Tapeout-Tools:Tapeout-Tools` section.
Examples of transforms
----------------------
There are multiple examples of transforms that you can apply and are spread across the FIRRTL ecosystem.
Within FIRRTL there is a default set of supported transforms located in https://github.com/freechipsproject/firrtl/tree/master/src/main/scala/firrtl/transforms.
This includes transforms that can flatten modules (``Flatten``), group modules together (``GroupAndDedup``), and more.
Transforms can be standalone or can take annotations as input. Annotations are used to pass information between FIRRTL transforms. This includes information on
what modules to flatten, group, and more. Annotations can be added to the code by
adding them to your Chisel source or by creating a serialized annotation ``json`` file and adding it to the FIRRTL compiler
(note: annotating the Chisel source will automatically serialize the annotation as a ``json`` snippet into the build system for you).
**The recommended way to annotate something is to do it in the Chisel source, but not all annotation types have Chisel APIs**.
The example below shows two ways to annotate the signal using the ``DontTouchAnnotation``
(makes sure that a particular signal is not removed by the "Dead Code Elimination" pass in FIRRTL):
* use the Chisel API/wrapper function called ``dontTouch`` that does this automatically for you (more `dontTouch <https://www.chisel-lang.org/api/SNAPSHOT/chisel3/dontTouch$.html>`__ information):
* directly annotate the signal with the ``annotate`` function and the ``DontTouchAnnotation`` class if there is no Chisel API for it (note: most FIRRTL annotations have Chisel APIs for them)
.. code-block:: scala
class TopModule extends Module {
...
val submod = Module(new Submodule)
...
}
class Submodule extends Module {
...
val some_signal := ...
// MAIN WAY TO USE `dontTouch`
// how to annotate if there is a Chisel API/wrapper
chisel3.dontTouch(some_signal)
// how to annotate WITHOUT a Chisel API/wrapper
annotate(new ChiselAnnotation {
def toFirrtl = DontTouchAnnotation(some_signal.toNamed)
})
...
}
Here is an example of the ``DontTouchAnnotation`` when it is serialized:
.. code-block:: json
[
{
"class": "firrtl.transforms.DontTouchAnnotation",
"target": "~TopModule|Submodule>some_signal"
}
]
In this case, the specific syntax depends on the type of annotation and its fields.
One of the easier ways to figure out the serialized syntax is to first try and find a Chisel
annotation to add to the code. Then you can look at the collateral that is generated from the
build system, find the ``*.anno.json``, and find the proper syntax for the annotation.
Once ``yourAnnoFile.json`` is created then you can add ``-faf yourAnnoFile.json`` to the FIRRTL
compiler invocation in ``common.mk``.
.. literalinclude:: ../../common.mk
:language: make
:start-after: DOC include start: FirrtlCompiler
:end-before: DOC include end: FirrtlCompiler
If you are interested in writing FIRRTL transforms please refer to the FIRRTL documentation located here:
https://github.com/freechipsproject/firrtl/wiki.

3
docs/Customization/index.rst
View File

@@ -29,8 +29,6 @@ We also provide information on:
- The boot process for Chipyard SoCs
- Examples of FIRRTL transforms used in Chipyard, and where they are specified
We recommend reading all these pages in order. Hit next to get started!
.. toctree::
@@ -50,5 +48,4 @@ We recommend reading all these pages in order. Hit next to get started!
Incorporating-Verilog-Blocks
Memory-Hierarchy
Boot-Process
Firrtl-Transforms
IOBinders

2
docs/Tools/FIRRTL.rst
View File

@@ -7,6 +7,4 @@ Without going into too much detail, FIRRTL is consumed by FIRRTL compilers which
An example of a FIRRTL pass (transformation) is one that optimizes out unused signals.
Once the transformations are done, a Verilog file is emitted and the build process is done.
To see how FIRRTL is transformed to Verilog in Chipyard, please visit the :ref:`firrtl-transforms` section.
For more information on FIRRTL, please visit their `website <https://chisel-lang.org/firrtl/>`__.

26
tools/tapeout/src/main/scala/transforms/ExtraTransforms.scala
View File

@@ -1,26 +0,0 @@
// See LICENSE for license details.
package tapeout.transforms
import firrtl.Mappers._
import firrtl._
import firrtl.annotations.{CircuitTarget, ModuleTarget, SingleTargetAnnotation}
import firrtl.ir._
import firrtl.stage.Forms
import firrtl.stage.TransformManager.TransformDependency
import firrtl.options.{Dependency}
class ExtraLowTransforms extends Transform with DependencyAPIMigration {
// this PropagatePresetAnnotations is needed to run the RemoveValidIf pass (that is removed from CIRCT).
// additionally, since that pass isn't explicitly a prereq of the LowFormEmitter it
// needs to wrapped in this xform
override def prerequisites: Seq[TransformDependency] = Forms.LowForm :+
Dependency[firrtl.transforms.PropagatePresetAnnotations]
override def optionalPrerequisites: Seq[TransformDependency] = Forms.LowFormOptimized
override def optionalPrerequisiteOf: Seq[TransformDependency] = Forms.LowEmitters
override def invalidates(a: Transform): Boolean = false
def execute(state: CircuitState): CircuitState = {
state
}
}

51
tools/tapeout/src/main/scala/transforms/GenerateModelStageMain.scala
View File

@@ -1,51 +0,0 @@
package tapeout.transforms
import tapeout.transforms.stage._
import firrtl._
import firrtl.annotations._
import firrtl.ir._
import firrtl.options.{Dependency, InputAnnotationFileAnnotation, StageMain}
import firrtl.stage.{FirrtlCircuitAnnotation, FirrtlStage, RunFirrtlTransformAnnotation}
import logger.LazyLogging
private class GenerateModelStageMain(annotations: AnnotationSeq) extends LazyLogging {
val outAnno: Option[String] = annotations.collectFirst { case OutAnnoAnnotation(s) => s }
val annoFiles: List[String] = annotations.flatMap {
case InputAnnotationFileAnnotation(f) => Some(f)
case _ => None
}.toList
// Dump firrtl and annotation files
// Use global param outAnno
protected def dumpAnnos(
annotations: AnnotationSeq
): Unit = {
outAnno.foreach { annoPath =>
val outputFile = new java.io.PrintWriter(annoPath)
outputFile.write(JsonProtocol.serialize(annotations.filter(_ match {
case _: DeletedAnnotation => false
case _: EmittedComponent => false
case _: EmittedAnnotation[_] => false
case _: FirrtlCircuitAnnotation => false
case _: OutAnnoAnnotation => false
case _ => true
})))
outputFile.close()
}
}
def executeStageMain(): Unit = {
val annos = new FirrtlStage().execute(Array.empty, annotations)
annos.collectFirst { case FirrtlCircuitAnnotation(circuit) => circuit } match {
case Some(circuit) =>
dumpAnnos(annos)
case _ =>
throw new Exception(s"executeStageMain failed while executing FIRRTL!\n")
}
}
}
// main run class
object GenerateModelStageMain extends StageMain(new TapeoutStage())

48
tools/tapeout/src/main/scala/transforms/retime/Retime.scala
View File

@@ -1,48 +0,0 @@
// See LICENSE for license details.
package tapeout.transforms.retime
import chisel3.experimental.RunFirrtlTransform
import firrtl.annotations._
import firrtl.stage.Forms
import firrtl.stage.TransformManager.TransformDependency
import firrtl.{CircuitState, DependencyAPIMigration, Transform}
case class RetimeAnnotation(target: Named) extends SingleTargetAnnotation[Named] {
override def duplicate(n: Named): Annotation = RetimeAnnotation(n)
}
class RetimeTransform extends Transform with DependencyAPIMigration {
override def prerequisites: Seq[TransformDependency] = Forms.LowForm
override def optionalPrerequisites: Seq[TransformDependency] = Forms.LowFormOptimized
override def optionalPrerequisiteOf: Seq[TransformDependency] = Forms.LowEmitters
override def invalidates(a: Transform): Boolean = false
override def execute(state: CircuitState): CircuitState = {
state.annotations.filter(_.isInstanceOf[RetimeAnnotation]) match {
case Nil => state
case seq =>
seq.foreach {
case RetimeAnnotation(ModuleName(module, CircuitName(_))) =>
logger.info(s"Retiming module $module")
case RetimeAnnotation(ComponentName(name, ModuleName(module, CircuitName(_)))) =>
logger.info(s"Retiming instance $module.$name")
case _ =>
throw new Exception(s"There should be RetimeAnnotations, got ${seq.mkString(" -- ")}")
}
state
}
}
}
trait RetimeLib {
self: chisel3.Module =>
def retime[T <: chisel3.Module](module: T): Unit = {
chisel3.experimental.annotate(new chisel3.experimental.ChiselAnnotation with RunFirrtlTransform {
def transformClass: Class[_ <: Transform] = classOf[RetimeTransform]
def toFirrtl: Annotation = RetimeAnnotation(module.toNamed)
})
}
}

50
tools/tapeout/src/main/scala/transforms/stage/TapeoutStage.scala
View File

@@ -1,50 +0,0 @@
// See LICENSE for license details.
package tapeout.transforms.stage
import tapeout.transforms.GenerateModelStageMain
import chisel3.stage.ChiselCli
import firrtl.stage.{RunFirrtlTransformAnnotation}
import firrtl.AnnotationSeq
import firrtl.annotations.{Annotation, NoTargetAnnotation}
import firrtl.options.{HasShellOptions, Shell, ShellOption, Stage, Unserializable}
import firrtl.stage.FirrtlCli
import logger.Logger
sealed trait TapeoutOption extends Unserializable {
this: Annotation =>
}
case class OutAnnoAnnotation(outAnno: String) extends NoTargetAnnotation with TapeoutOption
object OutAnnoAnnotation extends HasShellOptions {
val options: Seq[ShellOption[_]] = Seq(
new ShellOption[String](
longOption = "out-anno-file",
shortOption = Some("oaf"),
toAnnotationSeq = (s: String) => Seq(OutAnnoAnnotation(s)),
helpText = "out-anno-file"
)
)
}
trait TapeoutCli {
this: Shell =>
parser.note("Tapeout specific options")
Seq(
OutAnnoAnnotation
).foreach(_.addOptions(parser))
}
class TapeoutStage() extends Stage {
override val shell: Shell = new Shell(applicationName = "tapeout") with TapeoutCli with ChiselCli with FirrtlCli
override def run(annotations: AnnotationSeq): AnnotationSeq = {
Logger.makeScope(annotations) {
val stageMain = new GenerateModelStageMain(annotations)
stageMain.executeStageMain()
}
annotations
}
}

79
tools/tapeout/src/main/scala/transforms/utils/FileUtils.scala
View File

@@ -1,79 +0,0 @@
// See LICENSE for license details.
package tapeout.transforms.utils
import chisel3.experimental.{annotate, ChiselAnnotation}
import firrtl._
import firrtl.annotations._
import firrtl.stage.Forms
import firrtl.stage.TransformManager.TransformDependency
import firrtl.transforms.BlackBoxTargetDirAnno
object WriteConfig {
def apply(dir: String, file: String, contents: String): Unit = {
val writer = new java.io.PrintWriter(new java.io.File(s"$dir/$file"))
writer.write(contents)
writer.close()
}
}
object GetTargetDir {
def apply(state: CircuitState): String = {
val annos = state.annotations
val destDir = annos.map {
case BlackBoxTargetDirAnno(s) => Some(s)
case _ => None
}.flatten
val loc = {
if (destDir.isEmpty) "."
else destDir.head
}
val targetDir = new java.io.File(loc)
if (!targetDir.exists()) FileUtils.makeDirectory(targetDir.getAbsolutePath)
loc
}
}
trait HasSetTechnologyLocation {
self: chisel3.Module =>
def setTechnologyLocation(dir: String) {
annotate(new ChiselAnnotation {
override def toFirrtl: Annotation = {
TechnologyLocationAnnotation(dir)
}
})
}
}
case class TechnologyLocationAnnotation(dir: String) extends SingleTargetAnnotation[CircuitName] {
val target: CircuitName = CircuitName("All")
override def duplicate(n: CircuitName): Annotation = TechnologyLocationAnnotation(dir)
}
class TechnologyLocation extends Transform with DependencyAPIMigration {
override def prerequisites: Seq[TransformDependency] = Forms.LowForm
override def optionalPrerequisites: Seq[TransformDependency] = Forms.LowFormOptimized
override def optionalPrerequisiteOf: Seq[TransformDependency] = Forms.LowEmitters
def execute(state: CircuitState): CircuitState = {
throw new Exception("Technology Location transform execution doesn't work!")
}
def get(state: CircuitState): String = {
val annos = state.annotations
val dir = annos.flatMap {
case TechnologyLocationAnnotation(dir) => Some(dir)
case _ => None
}
dir.length match {
case 0 => ""
case 1 =>
val targetDir = new java.io.File(dir.head)
if (!targetDir.exists()) throw new Exception(s"Technology yaml directory $targetDir doesn't exist!")
dir.head
case _ => throw new Exception("Only 1 tech directory annotation allowed!")
}
}
}

5
tools/tapeout/src/main/scala/transforms/utils/LowerAnnotations.scala
View File

@@ -1,5 +0,0 @@
package tapeout.transforms.utils
object LowerName {
def apply(s: String): String = s.replace(".", "_").replace("[", "_").replace("]", "")
}

27
tools/tapeout/src/main/scala/transforms/utils/ProgrammaticBundle.scala
View File

@@ -1,27 +0,0 @@
package tapeout.transforms.utils
import chisel3._
import scala.collection.immutable.ListMap
class CustomBundle[T <: Data](elts: (String, T)*) extends Record {
val elements = ListMap(elts.map { case (field, elt) => field -> chiselTypeOf(elt) }: _*)
def apply(elt: String): T = elements(elt)
def apply(elt: Int): T = elements(elt.toString)
}
class CustomIndexedBundle[T <: Data](elts: (Int, T)*) extends Record {
// Must be String, Data
val elements = ListMap(elts.map { case (field, elt) => field.toString -> chiselTypeOf(elt) }: _*)
// TODO: Make an equivalent to the below work publicly (or only on subclasses?)
def indexedElements = ListMap(elts.map { case (field, elt) => field -> chiselTypeOf(elt) }: _*)
def apply(elt: Int): T = elements(elt.toString)
}
object CustomIndexedBundle {
def apply[T <: Data](gen: T, idxs: Seq[Int]) = new CustomIndexedBundle(idxs.map(_ -> gen): _*)
// Allows Vecs of elements of different types/widths
def apply[T <: Data](gen: Seq[T]) = new CustomIndexedBundle(gen.zipWithIndex.map { case (elt, field) =>
field -> elt
}: _*)
}

23
tools/tapeout/src/main/scala/transforms/utils/YamlHelpers.scala
View File

@@ -1,23 +0,0 @@
package tapeout.transforms.utils
import firrtl.FileUtils
import net.jcazevedo.moultingyaml._
import java.io.File
class YamlFileReader(resource: String) {
def parse[A](file: String = "")(implicit reader: YamlReader[A]): Seq[A] = {
// If the user doesn't provide a Yaml file name, use defaults
val yamlString = file match {
case f if f.isEmpty =>
// Use example config if no file is provided
val stream = FileUtils.getTextResource(resource)
stream
case f if new File(f).exists =>
FileUtils.getText(f)
case _ =>
throw new Exception("No valid Yaml file found!")
}
yamlString.parseYamls.map(x => reader.read(x))
}
}

8
variables.mk
View File

@@ -9,7 +9,6 @@ HELP_COMPILATION_VARIABLES = \
" SBT_OPTS = set additional sbt command line options (these take the form -Dsbt.<option>=<setting>) " \
" See https://www.scala-sbt.org/1.x/docs/Command-Line-Reference.html\#Command+Line+Options" \
" SBT = if overridden, used to invoke sbt (default is to invoke sbt by sbt-launch.jar)" \
" FIRRTL_LOGLEVEL = if overridden, set firrtl log level (default is error)"
HELP_PROJECT_VARIABLES = \
" SUB_PROJECT = use the specific subproject default variables [$(SUB_PROJECT)]" \
@@ -176,11 +175,6 @@ CHISEL_LOG_FILE ?= $(build_dir)/$(long_name).chisel.log
MFC_EXTRA_ANNO_FILE ?= $(build_dir)/$(long_name).extrafirtool.anno.json
FINAL_ANNO_FILE ?= $(build_dir)/$(long_name).appended.anno.json
# scala firrtl compiler (sfc) outputs
SFC_FIRRTL_BASENAME ?= $(build_dir)/$(long_name).sfc
SFC_FIRRTL_FILE ?= $(SFC_FIRRTL_BASENAME).fir
SFC_ANNO_FILE ?= $(build_dir)/$(long_name).sfc.anno.json
# firtool compiler outputs
MFC_TOP_HRCHY_JSON ?= $(build_dir)/top_module_hierarchy.json
MFC_MODEL_HRCHY_JSON ?= $(build_dir)/model_module_hierarchy.json
@@ -263,8 +257,6 @@ define run_sbt_assembly
cd $(base_dir) && $(SBT) ";project $(1); set assembly / assemblyOutputPath := file(\"$(2)\"); assembly" && touch $(2)
endef
FIRRTL_LOGLEVEL ?= error
#########################################################################################
# output directory for tests
#########################################################################################