403 lines
16 KiB
ReStructuredText
403 lines
16 KiB
ReStructuredText
Targets
|
|
=======
|
|
|
|
FireSim generates SoC models by transforming RTL emitted by a Chisel
|
|
generator, such as the Rocket SoC generator. Subject to
|
|
conditions outlined in :ref:`rtl-restrictions`, if it can be
|
|
generated by Chisel, it can be simulated in FireSim.
|
|
|
|
.. _rtl-restrictions:
|
|
|
|
Restrictions on Target RTL
|
|
--------------------------
|
|
|
|
Current limitations in Golden Gate place the following restrictions on the (FIR)RTL that can be
|
|
transformed and thus used in FireSim:
|
|
|
|
#. The top-level module must have no inputs or outputs. Input stimulus and output capture must be
|
|
implemented using target RTL or target-to-host Bridges.
|
|
#. All target clocks must be generated by a single ``RationalClockBridge``.
|
|
#. Black boxes must be "clock-gateable" by replacing its input clock with a gated equivalent which will be used
|
|
to stall simulation time in that module.
|
|
|
|
a. As a consequence, target clock-gating cannot be implemented using black-box primitives, and must instead be modeled by
|
|
adding clock-enables to all state elements of the gated clock domain (i.e., by adding an enable or feedback mux on registers to
|
|
conditionally block updates, and by gating write-enables on memories).
|
|
#. Asynchronous reset must only be implemented using Rocket Chip's black-box async reset.
|
|
These are replaced with synchronously reset registers using a FIRRTL transformation.
|
|
|
|
.. _verilog-ip:
|
|
|
|
--------------------
|
|
Including Verilog IP
|
|
--------------------
|
|
|
|
FireSim now supports target designs that incorporate Verilog IP using the standard ``BlackBox``
|
|
`interface from Chisel <https://github.com/freechipsproject/chisel3/wiki/Blackboxes>`_. For an
|
|
example of how to add Verilog IP to a target system based on Rocket Chip, see the `Incorporating
|
|
Verilog Blocks
|
|
<https://chipyard.readthedocs.io/en/latest/Customization/Incorporating-Verilog-Blocks.html>`_
|
|
section of the Chipyard documentation.
|
|
|
|
#. For the transform to work, the Chisel Blackbox that wraps the Verilog IP must have input clocks
|
|
that can safely be clock-gated.
|
|
#. The compiler that produces the decoupled simulator ("FAME Transform") automatically recognizes
|
|
such blackboxes inside the target design.
|
|
#. The compiler automatically gates each clock of the Verilog IP to ensure that it deterministically
|
|
advances in lockstep with the rest of the simulator.
|
|
#. This allows any Verilog module, subject to the constraint above, to be instantiated anywhere in the target
|
|
design using the standard Chisel Blackbox interface.
|
|
|
|
----------------------------
|
|
Multiple Clock Domains
|
|
----------------------------
|
|
|
|
FireSim can support simulating targets that have multiple clock
|
|
domains. As stated above, all clocks must be generated using a single
|
|
``RationalClockBridge``. For most users the default FireSim test harness in Chipyard will suffice,
|
|
if you need to define a custom test harness instantiate the ``RationalClockBridge`` like so:
|
|
|
|
.. literalinclude:: ../../sim/src/main/scala/midasexamples/TrivialMulticlock.scala
|
|
:language: scala
|
|
:start-after: DOC include start: RationalClockBridge Usage
|
|
:end-before: DOC include end: RationalClockBridge Usage
|
|
|
|
|
|
Further documentation can be found in the source
|
|
(``sim/midas/src/main/scala/midas/widgets/ClockBridge.scala``).
|
|
|
|
=================
|
|
The Base Clock
|
|
=================
|
|
By convention, target time is specified in cycles of the `base clock`,
|
|
which is defined to be the clock of the ``RationalClockBridge`` whose clock ratio (multiplier / divisor)
|
|
is one. While we suggest making the base clock the fastest clock in your system, as in any
|
|
microprocessor-based system it will likely correspond to your core clock
|
|
frequency, this is not a constraint.
|
|
|
|
============
|
|
Limitations:
|
|
============
|
|
* The number of target clocks FireSim can simulate is bounded by the number of BUFGCE resources
|
|
available on the host FPGA, as these are used to independently clock-gate each target clock.
|
|
* As its name suggests, the ``RationalClockBridge`` can only generate target clocks
|
|
that are rationally related. Specifically, all requested frequencies must be
|
|
expressable in the form:
|
|
|
|
.. math::
|
|
f_{i} = \frac{f_{lcm}}{k_{i}}
|
|
|
|
Where,
|
|
* :math:`f_{i}` is the desired frequency of the :math:`i^{th}` clock
|
|
* :math:`f_{lcm}`, is the least-common multiple of all requested frequencies
|
|
* :math:`k_{i}` is a 16-bit unsigned integer
|
|
|
|
|
|
An arbitrary frequency can be modeled using a sufficiently precise rational
|
|
multiple. Golden Gate will raise a compile-time error if it cannot support
|
|
a desired frequency.
|
|
* Each bridge module must reside entirely within a single clock domain. The Bridge's target interface
|
|
must contain a single input clock, and all inputs and outputs of the
|
|
bridge module must be latched and launched, respectively, by registers in the
|
|
same clock domain.
|
|
|
|
.. _generating-different-targets:
|
|
|
|
Target-Side FPGA Constraints
|
|
----------------------------
|
|
|
|
FireSim provides utilities to generate Xilinx Design Constraints (XDC) from
|
|
string snippets in target's Chisel. Golden Gate collects these annotations and
|
|
emits separate xdc files for synthesis and implementation. See
|
|
:ref:`fpga-build-files` for a complete listing of output files used in FPGA
|
|
compilation.
|
|
|
|
-----------------------------
|
|
RAM Inference Hints
|
|
-----------------------------
|
|
|
|
Vivado generally does a reasonable job inferring embedded memories from
|
|
FireSim-generated RTL, though there are some cases in which it must be coaxed. For example:
|
|
|
|
* Due to insufficient BRAM resources, you may wish to use URAM for a memory that'd infer as BRAM.
|
|
* If Vivado can't find pipeline registers to absorb into a URAM or none exist in the target, you may get an warning like::
|
|
|
|
[Synth 8-6057] Memory: "<memory>" defined in module: "<module>" implemented as Ultra-Ram
|
|
has no pipeline registers. It is recommended to use pipeline registers to achieve high
|
|
performance.
|
|
|
|
Since Golden Gate modifies the module hierarchy extensively, it's highly
|
|
desirable to annotate these memories in the Chisel source so that their hints
|
|
may move with the memory instance. This is a more robust alternative to
|
|
relying on wildcard / glob matches from a static XDC specification.
|
|
|
|
Chisel memories can be annotated *in situ* like so:
|
|
|
|
.. literalinclude:: ../../sim/midas/targetutils/src/test/scala/RAMStyleHintSpec.scala
|
|
:language: scala
|
|
:start-after: DOC include start: Basic RAM Hint
|
|
:end-before: DOC include end: Basic RAM Hint
|
|
|
|
Alternatively, you can "dot-in" (traverse public members of a Scala class
|
|
hierarchy) to annotate a memory in a submodule. Here's an example:
|
|
|
|
.. literalinclude:: ../../sim/midas/targetutils/src/test/scala/RAMStyleHintSpec.scala
|
|
:language: scala
|
|
:start-after: DOC include start: RAM Hint From Parent
|
|
:end-before: DOC include end: RAM Hint From Parent
|
|
|
|
These annotations can be deployed anywhere: in the target, in bridge modules,
|
|
and in internal FireSim RTL. The resulting constraints should appear the
|
|
synthesis xdc file emitted by Golden Gate. For more information see the ScalaDoc
|
|
for ``RAMStyleHint`` or read the source located at:
|
|
:gh-file-ref:`sim/midas/targetutils/src/main/scala/midas/xdc/RAMStyleHint.scala`.
|
|
|
|
Provided Target Designs
|
|
-----------------------
|
|
|
|
-----------------------------
|
|
Target Generator Organization
|
|
-----------------------------
|
|
|
|
FireSim provides multiple `projects`, each for a different type of target. Each
|
|
project has its own chisel generator that invokes Golden Gate, its own driver
|
|
sources, and a makefrag that plugs into the Make-based build system that
|
|
resides in ``sim/``. These projects are:
|
|
|
|
1. **firesim** (Default): rocket chip-based targets. These include targets with
|
|
either BOOM or rocket pipelines, and should be your starting point if you're
|
|
building an SoC with the Rocket Chip generator.
|
|
2. **midasexamples**: the Golden Gate example designs. Located at
|
|
:gh-file-ref:`sim/src/main/scala/midasexamples`, these are a set of simple chisel
|
|
circuits like GCD, that demonstrate how to use Golden Gate. These are useful test
|
|
cases for bringing up new Golden Gate features.
|
|
3. **bridges**: tests for firesim-lib bridges. These have more dependencies and
|
|
involve more logic than `midasexamples`.
|
|
4. **fasedtests**: designs to do integration testing of FASED memory-system timing models.
|
|
|
|
Projects have the following directory structure:
|
|
|
|
.. code-block:: text
|
|
|
|
sim/
|
|
├-Makefile # Top-level makefile for projects where FireSim is the top-level repo
|
|
├-Makefrag # Target-agnostic makefrag, with recipes to generate drivers and RTL simulators
|
|
├-src/main/scala/{target-project}/
|
|
│ └─Makefrag # Defines target-specific make variables and recipes.
|
|
├-src/main/cc/{target-project}/
|
|
│ ├─{driver-csrcs}.cc # The target's simulation driver, and sofware-model sources
|
|
│ └─{driver-headers}.h
|
|
└-src/main/makefrag/{target-project}/
|
|
├─Generator.scala # Contains the main class that generates target RTL and calls Golden Gate
|
|
└─{other-scala-sources}.scala
|
|
|
|
----------------------------
|
|
Specifying A Target Instance
|
|
----------------------------
|
|
|
|
To generate a specific instance of a target, the build system leverages five Make variables:
|
|
|
|
1. ``TARGET_PROJECT``: this points the Makefile (`sim/Makefile`) at the right
|
|
target-specific Makefrag, which defines the generation and metasimulation
|
|
software recipes. The makefrag for the default target project is
|
|
defined at ``sim/src/main/makefrag/firesim``.
|
|
|
|
2. ``DESIGN``: the name of the top-level Chisel module to generate (a Scala class name). These are defined
|
|
in FireChip Chipyard generator.
|
|
|
|
3. ``TARGET_CONFIG``: specifies a ``Config`` instance that is consumed by the target design's
|
|
generator. For the default firesim target project, predefined configs are described in
|
|
in the FireChip Chipyard generator.
|
|
|
|
4. ``PLATFORM_CONFIG``: specifies a ``Config`` instance that is consumed by
|
|
Golden Gate and specifies compiler-level and host-land
|
|
parameters, such as whether to enable assertion synthesis, or multi-ported RAM optimizations.
|
|
Common platform configs are described in ``firesim-lib/sim/src/main/scala/configs/CompilerConfigs.scala``).
|
|
|
|
5. ``PLATFORM``: this points the Makefile (`sim/Makefile`) at the right
|
|
FPGA platform to build for. This must correspond to a platform
|
|
defined at :gh-file-ref:`platforms`.
|
|
|
|
``TARGET_CONFIG`` and ``PLATFORM_CONFIG`` are strings that are used to construct a
|
|
``Config`` instance (derives from RocketChip's parameterization system, ``Config``, see the
|
|
`CDE repo
|
|
<https://github.com/chipsalliance/cde>`_). These strings are of the form
|
|
"{..._}{<Class Name>\_}<Class Name>". Only the final, base class name is
|
|
compulsory: class names that are prepended with "_" are used to create a
|
|
compound Config instance.
|
|
|
|
.. code-block:: scala
|
|
|
|
// Specify by setting TARGET_CONFIG=Base
|
|
class Base extends Config((site, here, up) => {...})
|
|
class Override1 extends Config((site, here, up) => {...})
|
|
class Override2 extends Config((site, here, up) => {...})
|
|
// Specify by setting TARGET_CONFIG=Compound
|
|
class Compound extends Config(new Override2 ++ new Override1 ++ new Base)
|
|
// OR by setting TARGET_CONFIG=Override2_Override1_Base
|
|
// Can specify undefined classes this way. ex: TARGET_CONFIG=Override2_Base
|
|
|
|
With this scheme, you don't need to define a Config class for every instance you
|
|
wish to generate, making it very useful for sweeping over a parameterization space.
|
|
|
|
**Note that the precedence of Configs decreases from left to right in a
|
|
string**. Appending a config to an existing one will only have an effect if it
|
|
sets a field not already set in higher precendence Configs. For example,
|
|
"BaseF1Config_SetFieldAtoX" is equivalent to
|
|
"BaseF1Config_SetFieldAtoX_SetFieldAtoY".
|
|
|
|
How a particular Config resolves it's ``Field`` s can be unintuitive for complex
|
|
compound ``Config`` s. One precise way to check a config is doing what you
|
|
expect is to open the scala REPL, instantiate an instance of the desired
|
|
``Config``, and inspect its fields.
|
|
|
|
.. code-block:: shell
|
|
|
|
$ make sbt # Launch into SBT's shell with extra FireSim arguments
|
|
|
|
sbt:firechip> console # Launch the REPL
|
|
|
|
scala> val inst = (new firesim.firesim.FireSimRocketChipConfig).toInstance # Make an instance
|
|
|
|
inst: freechips.rocketchip.config.Config = FireSimRocketChipConfig
|
|
|
|
scala> import freechips.rocketchip.subsystem._ # Get some important Fields
|
|
|
|
import freechips.rocketchip.subsystem.RocketTilesKey
|
|
|
|
scala> inst(RocketTilesKey).size # Query number of cores
|
|
|
|
res2: Int = 1
|
|
|
|
scala> inst(RocketTilesKey).head.dcache.get.nWays # Query L1 D$ associativity
|
|
|
|
res3: Int = 4
|
|
|
|
|
|
Rocket Chip Generator-based SoCs (firesim project)
|
|
--------------------------------------------------
|
|
|
|
Using the Make variables listed above, we give examples of generating different targets using
|
|
the default Rocket Chip-based target project.
|
|
|
|
-----------------
|
|
Rocket-based SoCs
|
|
-----------------
|
|
|
|
Three design classes use Rocket scalar in-order pipelines.
|
|
|
|
Single core, Rocket pipeline (default)
|
|
|
|
.. code-block:: bash
|
|
|
|
make TARGET_CONFIG=FireSimRocketConfig
|
|
|
|
|
|
Single-core, Rocket pipeline, with network interface
|
|
|
|
.. code-block:: bash
|
|
|
|
make TARGET_CONFIG=WithNIC_FireSimRocketChipConfig
|
|
|
|
|
|
Quad-core, Rocket pipeline
|
|
|
|
.. code-block:: bash
|
|
|
|
make TARGET_CONFIG=FireSimQuadRocketConfig
|
|
|
|
|
|
---------------
|
|
BOOM-based SoCs
|
|
---------------
|
|
|
|
The BOOM (`Berkeley Out-of-Order Machine <https://github.com/ucb-bar/riscv-boom>`_) superscalar out-of-order pipelines can also be used with the same design classes that the Rocket pipelines use. Only the TARGET_CONFIG needs to be changed, as shown below:
|
|
|
|
Single-core BOOM
|
|
|
|
.. code-block:: bash
|
|
|
|
make TARGET_CONFIG=FireSimLargeBoomConfig
|
|
|
|
Single-core BOOM, with network interface
|
|
|
|
.. code-block:: bash
|
|
|
|
make TARGET_CONFIG=WithNIC_FireSimBoomConfig
|
|
|
|
|
|
----------------------------------------------------------
|
|
Generating A Different FASED Memory-Timing Model Instance
|
|
----------------------------------------------------------
|
|
|
|
Golden Gate's memory-timing model generator, FASED, can elaborate a space of
|
|
different DRAM model instances: we give some typical ones here. These targets
|
|
use the Makefile-defined defaults of ``DESIGN=FireSim PLATFORM_CONFIG=BaseF1Config``.
|
|
|
|
Quad-rank DDR3 first-ready, first-come first-served memory access scheduler
|
|
|
|
.. code-block:: bash
|
|
|
|
make TARGET_CONFIG=DDR3FRFCFS_FireSimRocketConfig
|
|
|
|
|
|
As above, but with a 4 MiB (maximum simulatable capacity) last-level-cache model
|
|
|
|
.. code-block:: bash
|
|
|
|
make TARGET_CONFIG=DDR3FRFCFSLLC4MB_FireSimRocketConfig
|
|
|
|
FASED *timing-model* configurations are passed to the FASED Bridges
|
|
in your Target's FIRRTL, and so must be prepended to ``TARGET_CONFIG``.
|
|
|
|
|
|
Midas Examples (midasexamples project)
|
|
--------------------------------------------------
|
|
This project can generate a handful of toy target-designs (set with the make
|
|
variable ``DESIGN``). Each of these designs has their own chisel source file and serves to demostrate
|
|
the features of Golden Gate.
|
|
|
|
Some notable examples are:
|
|
|
|
#. ``GCD``: the "Hello World!" of hardware.
|
|
#. ``WireInterconnect``: demonstrates how combinational paths can be modeled with Golden Gate.
|
|
#. ``PrintfModule``: demonstrates synthesizable printfs
|
|
#. ``AssertModule``: demonstrates synthesizable assertions
|
|
|
|
To generate a target, set the make variable
|
|
``TARGET_PROJECT=midasexamples``. so that the right project makefrag is
|
|
sourced.
|
|
|
|
--------
|
|
Examples
|
|
--------
|
|
|
|
To generate the GCD midasexample:
|
|
|
|
.. code-block:: bash
|
|
|
|
make DESIGN=GCD TARGET_PROJECT=midasexamples
|
|
|
|
FASED Tests (fasedtests project)
|
|
--------------------------------------------------
|
|
This project generates target designs capable of driving considerably more
|
|
bandwidth to an AXI4-memory slave than current FireSim targets. These are used to do
|
|
integration and stress testing of FASED instances.
|
|
|
|
--------
|
|
Examples
|
|
--------
|
|
|
|
Generate a synthesizable AXI4Fuzzer (based off of Rocket Chip's TL fuzzer), driving a
|
|
DDR3 FR-FCFS-based FASED instance.
|
|
|
|
.. code-block:: bash
|
|
|
|
make TARGET_PROJECT=fasedtests DESIGN=AXI4Fuzzer TARGET_CONFIG=FRFCFSConfig
|
|
|
|
As above, now configured to drive 10 million transactions through the instance.
|
|
|
|
.. code-block:: bash
|
|
|
|
make TARGET_PROJECT=fasedtests DESIGN=AXI4Fuzzer TARGET_CONFIG=NT10e7_FRFCFSConfig
|