replica
/
firesim
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

more docs

This commit is contained in:
Sagar Karandikar 2018-11-28 21:33:37 +00:00
parent 873c72abc9
commit d8095dc402
1 changed files with 37 additions and 27 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

64
docs/Advanced-Usage/Supernode.rst
View File

@ -1,31 +1,34 @@
Supernode
===============
Supernode - Multiple Simulations Per FPGA
============================================
Supernode is designed to improve FPGA resource utilization for smaller designs
and allow realistic rack topology simulation (32 simulated nodes) using a
single ``f1.16xlarge`` instance. Supernode requires slight changes in build and
runtime configurations. Supernode is currently only enabled for RocketChip
designs with NICs. More details about supernode can be found in the `FireSim
ISCA 2018 Paper <https://sagark.org/assets/pubs/firesim-isca2018.pdf>`__.
Supernode allows users to run multiple simulations per-FPGA in order to improve
FPGA resource utilization and reduce cost. For example, in the case of using
FireSim to simulate a datacenter scale system, supernode mode allows
realistic rack topology simulation (32 simulated nodes) using a
single ``f1.16xlarge`` instance (8 FPGAs).
Below, we outline the build and runtime configuration changes needed to utilize
supernode designs. Supernode is currently only enabled for RocketChip designs
with NICs. More details about supernode can be found in the `FireSim ISCA 2018
Paper <https://sagark.org/assets/pubs/firesim-isca2018.pdf>`__.
Intro
-----------
Supernode packs 4 identical designs into a single FPGA, and utilizes all 4 DDR
channels available for each FPGA on AWS F1 instances. It currently does so by
generating a wrapper top level target which encapsualtes the four simulated
target nodes. The packed nodes are treated as 4 separate nodes, are assigned their
own individual MAC addresses, and can perform any action a single node could:
run different programs, interact with each other over the network, utilize
different block device images, etc.
By default, supernode packs 4 identical designs into a single FPGA, and
utilizes all 4 DDR channels available on each FPGA on AWS F1 instances. It
currently does so by generating a wrapper top level target which encapsualtes
the four simulated target nodes. The packed nodes are treated as 4 separate
nodes, are assigned their own individual MAC addresses, and can perform any
action a single node could: run different programs, interact with each other
over the network, utilize different block device images, etc. In the networked
case, 4 separate network links are presented to the switch-side.
Build
Building
-----------
Since Supernode is currently impelmented as a wrapper top level config, most of
the relevant build components can be found in locations similar to target
components. Here, we outline some of the changes between supernode and regular
simulations.
Here, we outline some of the changes between supernode and regular
simulations that are required to build supernode designs.
The Supernode target configuration wrapper can be found in
``firesim/sim/src/main/scala/TargetConfigs.scala``. An example wrapper configuration is:
@ -37,9 +40,10 @@ The Supernode target configuration wrapper can be found in
new FireSimRocketChipConfig)
In this example, ``SupernodeFireSimRocketChipConfig`` is the wrapper, while
``FireSimRocketChipConfig`` is the target node configuration. Therefore, if we
want to simulate a different target configuration, we will generate a new
Supernode wrapper, with the new target configuration. For example:
``FireSimRocketChipConfig`` is the target node configuration. To
simulate a different target configuration, we will generate a new supernode
wrapper, with the new target configuration. For example, to simulate 4 quad-core
nodes on one FPGA, you can use:
::
@ -66,9 +70,10 @@ be selected the same as in regular FireSim configurations. For example:
We currently do not provide pre-built AGFIs for supernode. You must build your
own, using the supplied samples in ``config_build_recipes.ini``.
Importantly, in order to meet FPGA timing contraints, you must also manually
change the host clock frequency by editing the clock assignment in
change the host clock frequency by editing the clock assignment in
``firesim/platforms/f1/aws-fpga/hdk/cl/developer_designs/cl_firesim/design/cl_firesim.sv``.
This is done by change the following line from :
75MHz is a reasonable frequency for the supplied designs.
This is done by change the following line from:
::
assign firesim_internal_clock = clock_gend_90;
@ -97,11 +102,16 @@ Supernode topologies utilize a ``FireSimSuperNodeServerNode`` class in order to
represent one of the 4 simulated target nodes which also represents a single
FPGA mapping, while using a ``FireSimDummyServerNode`` class which represent
the other three simulated target nodes which do not represent an FPGA mapping.
Various example Supernode topologies are provided, ranging from 4 simulated
In supernode mode, topologies should always add nodes in pairs of 4, as one
``FireSimSuperNodeServerNode`` and three ``FireSimDummyServerNode`` s.
Various example Supernode topologies are provided, ranging from 4 simulated
target nodes to 1024 simulated target nodes.
Following are a couple of useful examples as templates for writing custom
Below are a couple of useful examples as templates for writing custom
Supernode topologies.
A sample Supernode topology of 4 simulated target nodes which can fit on a
single ``f1.2xlarge`` is: