From 6b01fc39fb255abbf51555b0099c88132a9a2009 Mon Sep 17 00:00:00 2001 From: Sagar Karandikar Date: Wed, 28 Jun 2023 15:54:25 +0000 Subject: [PATCH] update firesim basics --- docs/FireSim-Basics.rst | 178 ++++++++++-------- .../RHS-Research-Nitefury-II-FPGAs.rst | 12 ++ .../Xilinx-VCU118-FPGAs.rst | 12 ++ 3 files changed, 128 insertions(+), 74 deletions(-) create mode 100644 docs/Getting-Started-Guides/On-Premises-FPGA-Getting-Started/RHS-Research-Nitefury-II-FPGAs.rst create mode 100644 docs/Getting-Started-Guides/On-Premises-FPGA-Getting-Started/Xilinx-VCU118-FPGAs.rst diff --git a/docs/FireSim-Basics.rst b/docs/FireSim-Basics.rst index d11c26f5..ccb3b2a3 100644 --- a/docs/FireSim-Basics.rst +++ b/docs/FireSim-Basics.rst @@ -3,76 +3,101 @@ FireSim Basics =================================== -FireSim is a cycle-accurate, FPGA-accelerated scale-out computer system -simulation platform developed in the Berkeley Architecture Research Group in -the EECS Department at the University of California, Berkeley. +FireSim is an open-source +FPGA-accelerated full-system hardware simulation platform that makes +it easy to validate, profile, and debug RTL hardware implementations +at 10s to 100s of MHz. FireSim simplifies co-simulating +ASIC RTL with cycle-accurate hardware and software models for other system components (e.g. I/Os). FireSim can productively +scale from individual SoC simulations hosted on on-prem FPGAs (e.g., a single Xilinx Alveo board attached to a desktop) +to massive datacenter-scale simulations harnessing hundreds of cloud FPGAs (e.g., on Amazon EC2 F1). -FireSim is capable of simulating from **one to thousands of multi-core compute -nodes**, derived from **silicon-proven** and **open** target-RTL, with an optional -cycle-accurate network simulation tying them together. FireSim runs on FPGAs in **public -cloud** environments like AWS EC2 F1, removing the high capex traditionally -involved in large-scale FPGA-based simulation, as well as on on-premises FPGAs. +FireSim users across academia and industry (at 20+ institutions) have published +over 40 papers using FireSim in many areas, including computer architecture, +systems, networking, security, scientific computing, circuits, design +automation, and more (see the `Publications page `__ on +the FireSim website to learn more). FireSim +has also been used in the development of shipping commercial silicon. FireSim +was originally developed in the Electrical Engineering and Computer Sciences +Department at the University of California, Berkeley, but +now has industrial and academic contributors from all over the world. -FireSim is useful both for datacenter architecture research as well as running -many single-node architectural experiments in parallel on FPGAs. By harnessing -a standardized host platform and providing a large amount of -automation/tooling, FireSim drastically simplifies the process of building and -deploying large-scale FPGA-based hardware simulations. +This documentation will walk you through getting started with using FireSim and +serves as a reference for more advanced FireSim features. For higher-level +technical discussion about FireSim, see the `FireSim website `__. -To learn more, see the `FireSim website `__ and the FireSim -`ISCA 2018 paper `__. -For a two-minute overview that describes how FireSim simulates a datacenter, -see our ISCA 2018 lightning talk `on YouTube `__. +Three common FireSim usage models +--------------------------------------- -Three common use cases: --------------------------- +Below are three common usage models for FireSim. The first two are the most common, while the +third model is primarily for those interested in warehouse-scale computer research. The getting +started guides on this documentation site will cover all three models. -Single-Node Simulation In Parallel Using On-Premises FPGAs -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Single-Node Simulations Using One or More On-Premises FPGAs +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -In this mode, FireSim allows for simulation of individual Rocket -Chip-based nodes without a network, which allows individual simulations to run -at ~150 MHz. The FireSim manager has the ability to automatically distribute -jobs to on-premises FPGAs allowing users to harness existing FPGAs for quick turnaround time and -maximum flexibility. For example, users can run all of SPECInt2017 on Rocket Chip -in ~1 day by running the 10 separate workloads in parallel on 10 on-premises FPGAs. +In this usage model, FireSim allows for simulation of targets consisting of +individual SoC designs (e.g., those produced by `Chipyard +__`) at 150+ MHz running on on-premises +FPGAs, such as those attached to your local desktop, laptop, or cluster. Just +like on the cloud, the FireSim manager can automatically distribute and manage +jobs on one or more on-premises FPGAs, including running complex workloads like +SPECInt2017 with full reference inputs. -Single-Node Simulation In Parallel Using Cloud FPGAs +Single-Node Simulations Using Cloud FPGAs ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -In this mode, FireSim allows for simulation of individual Rocket -Chip-based nodes without a network, which allows individual simulations to run -at ~150 MHz. The FireSim manager has the ability to automatically distribute -jobs to many parallel simulations running on cloud FPGAs, expediting the process of running large -workloads like SPEC. For example, users can run all of SPECInt2017 on Rocket Chip -in ~1 day by running the 10 separate workloads in parallel on 10 FPGAs hosted in the cloud. +This usage model is similar to the previous on-premises case, but instead +deploys simulations on FPGAs attached to cloud instances, rather than requiring +users to obtain and set-up on-premises FPGAs. This allows for dynamically +scaling the number of FPGAs in-use to match workload requirements. For example, +it is just as cost effective to run the 10 workloads in SPECInt2017 in parallel +on 10 cloud FPGAs vs. running them serially on one cloud FPGA. -Datacenter/Cluster Simulation -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +All automation in FireSim works in both the on-premises and cloud +usage models, which enables a **hybrid usage model** where early development happens +on one (or a small cluster of) on-premises FPGA(s), while bursting to a large +number of cloud FPGAs when a high-degree of parallelism is necessary. + +Datacenter/Cluster Simulations on On-Premises or Cloud FPGAs +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ In this mode, FireSim also models a cycle-accurate network with -parameterizeable bandwidth and link latency, as well as configurable -topology, to accurately model current and future datacenter-scale +parameterizeable bandwidth, link latency, and configurable +topology to accurately model current and future datacenter-scale systems. For example, FireSim has been used to simulate 1024 quad-core -Rocket Chip-based nodes, interconnected by a 200 Gbps, 2us network. To learn +RISC-V Rocket Chip-based nodes, interconnected by a 200 Gbps, 2us Ethernet network. To learn more about this use case, see our `ISCA 2018 paper -`__ or `two-minute lightning talk -`__. +`__. + Other Use Cases --------------------- -This release does not support a non-cycle-accurate network as our `AWS Compute Blog Post/Demo -`__ -used. This feature will be restored in a future release. - -If you have other use-cases that we haven't covered, feel free to contact us! +If you have other use-cases that we haven't covered or don't fit into the above +buckets, feel free to contact us! Background/Terminology --------------------------- +Before we jump into setting up FireSim, it is important to clarify several terms +that we will use throughout the rest of this documentation. + +First, to disambiguate between the hardware being simulated and the computers doing +the simulating, we define: + +**Target** + The design and environment being simulated. Commonly, a + group of one or more RISC-V SoCs with or without a network between them. + +**Host** + The computers/FPGAs executing the FireSim simulation -- the **Run Farm** below. + +We frequently prefix words with these terms. For example, software can run +on the simulated RISC-V system (*target*-software) or on a host x86 machine (*host*-software). + + .. figure:: img/firesim_env.png :alt: FireSim Infrastructure Setup @@ -86,46 +111,46 @@ Background/Terminology command is just like the ``vagrant`` and ``docker`` commands, but for FPGA simulators instead of VMs/containers. -**Manager Instance** - This is the main host (ex. AWS EC2 instance or local machine) that you will - SSH-into and do work on. This is where you'll clone your copy of FireSim and - use the FireSim Manager to deploy builds/simulations from. -**Build Farm** - These are instances that are managed by the FireSim manager when you run FPGA builds. - The manager will automatically ship source for builds to these instances and - run the Verilog -> FPGA Image process on them. +Machines used to build and run FireSim simulations are broadly classified into +three groups: -**Run Farm** - These are a collection of instances that the manager - manages and deploys simulations onto. You can use multiple - Run Farms in parallel, to run multiple separate - simulations in parallel. +**Manager Machine/Instance** + This is the main host machine (e.g., your local desktop or an AWS EC2 + instance) that you will "do work" on. This is where you'll clone your copy of + FireSim and use the FireSim Manager to deploy builds/simulations from. -To disambiguate between the computers being simulated and the computers doing -the simulating, we also define: +**Build Farm Machines/Instances** + These are local machines ("build farm machines") or cloud instances ("build + farm instances") that are used by the FireSim manager to run FPGA bitstream + builds. The manager will automatically ship all sources necessary to run + builds to these machines and will run the Verilog to FPGA bitstream build + process on them. -**Target** - The design and environment under simulation. Generally, a - group of one or more multi-core RISC-V microprocessors with or without a network between them. +**Run Farm Machines/Instances** + These are a collection of local machines ("run farm machines") or cloud + instances ("run farm instances") with FPGAs attached that the manager manages + and deploys simulations onto. You can use multiple Run Farms in parallel to + run multiple separate simulations in parallel. -**Host** - The computers executing the FireSim simulation -- the **Run Farm** from above. -We frequently prefix words with these terms. For example, software can run -on the simulated RISC-V system (*target*-software) or on a host x86 machine (*host*-software). +In the simplest setup, a single host machine (e.g. your desktop) can serve +the function of all three of these: as the manager machine, the build farm +machine (assuming Vivado is installed), and the run farm machine (assuming +an FPGA is attached). + +One final piece of terminology will also be referenced throughout these +docs: **Golden Gate (MIDAS II)** The FIRRTL compiler used by FireSim to convert target RTL into a decoupled simulator. Formerly named MIDAS. -Get Started ------------ -FireSim supports many type of FPGAs and FPGA platforms! -Click one of the following links to get started with your particular platform. +Choose your platform to get started +-------------------------------------- -.. warning:: If using a Xilinx Alveo U250 or U280, we recommend the FPGA-specific flows instead of the Xilinx Vitis flow. +FireSim supports many types of FPGAs and FPGA platforms! Click one of the following links to work through the getting started guide for your particular platform. * :doc:`/Getting-Started-Guides/AWS-EC2-F1-Getting-Started/index` @@ -133,4 +158,9 @@ Click one of the following links to get started with your particular platform. * :doc:`/Getting-Started-Guides/On-Premises-FPGA-Getting-Started/Xilinx-Alveo-U280-FPGAs` -* :doc:`Getting-Started-Guides/On-Premises-FPGA-Getting-Started/Xilinx-Vitis-FPGAs` +* :doc:`/Getting-Started-Guides/On-Premises-FPGA-Getting-Started/Xilinx-VCU118-FPGAs` + +* :doc:`/Getting-Started-Guides/On-Premises-FPGA-Getting-Started/RHS-Research-Nitefury-II-FPGAs` + +* :doc:`Getting-Started-Guides/On-Premises-FPGA-Getting-Started/Xilinx-Vitis-FPGAs`. The Vitis flow is not recommended unless you have specific constraints that require using Vitis. Use the aforementioned `/Getting-Started-Guides/On-Premises-FPGA-Getting-Started/Xilinx-Alveo-U250-FPGAs` Guide instead. + diff --git a/docs/Getting-Started-Guides/On-Premises-FPGA-Getting-Started/RHS-Research-Nitefury-II-FPGAs.rst b/docs/Getting-Started-Guides/On-Premises-FPGA-Getting-Started/RHS-Research-Nitefury-II-FPGAs.rst new file mode 100644 index 00000000..079d4508 --- /dev/null +++ b/docs/Getting-Started-Guides/On-Premises-FPGA-Getting-Started/RHS-Research-Nitefury-II-FPGAs.rst @@ -0,0 +1,12 @@ +.. |fpga_name| replace:: RHS Research Nitefury II +.. |bit_type| replace:: ``bitstream_tar`` +.. |build_type| replace:: Xilinx Vivado + +.. include:: Intro-Template.rst + +.. toctree:: + :maxdepth: 3 + + Initial-Setup/Xilinx-Alveo-U250 + Running-Simulations/Running-Single-Node-Simulation-Xilinx-Alveo-U250 + Building-a-FireSim-Bitstream/Xilinx-Alveo-U250 diff --git a/docs/Getting-Started-Guides/On-Premises-FPGA-Getting-Started/Xilinx-VCU118-FPGAs.rst b/docs/Getting-Started-Guides/On-Premises-FPGA-Getting-Started/Xilinx-VCU118-FPGAs.rst new file mode 100644 index 00000000..97630ae1 --- /dev/null +++ b/docs/Getting-Started-Guides/On-Premises-FPGA-Getting-Started/Xilinx-VCU118-FPGAs.rst @@ -0,0 +1,12 @@ +.. |fpga_name| replace:: Xilinx VCU118 +.. |bit_type| replace:: ``bitstream_tar`` +.. |build_type| replace:: Xilinx Vivado + +.. include:: Intro-Template.rst + +.. toctree:: + :maxdepth: 3 + + Initial-Setup/Xilinx-Alveo-U250 + Running-Simulations/Running-Single-Node-Simulation-Xilinx-Alveo-U250 + Building-a-FireSim-Bitstream/Xilinx-Alveo-U250