Merge pull request #168 from firesim/dev
FireSim 1.5.0 Release (dev to master PR)
This commit is contained in:
Sagar Karandikar
GitHub
commit
7f20a19cec
|
|
@ -28,3 +28,9 @@
|
|||
[submodule "deploy/workloads/Speckle"]
|
||||
path = deploy/workloads/Speckle
|
||||
url = https://github.com/ccelio/Speckle.git
|
||||
[submodule "deploy/workloads/ccbench-cache-sweep/ccbench"]
|
||||
path = deploy/workloads/ccbench-cache-sweep/ccbench
|
||||
url = https://github.com/ucb-bar/ccbench
|
||||
[submodule "deploy/workloads/runscripts/gapbs-scripts/gapbs"]
|
||||
path = deploy/workloads/runscripts/gapbs-scripts/gapbs
|
||||
url = https://github.com/sbeamer/gapbs.git
|
||||
|
|
|
|||
61
CHANGELOG.md
61
CHANGELOG.md
|
|
@ -2,6 +2,67 @@
|
|||
|
||||
This changelog follows the format defined here: https://keepachangelog.com/en/1.0.0/
|
||||
|
||||
## [1.5.0] - 2019-02-24
|
||||
|
||||
A more detailed account of everything included is included in the dev->master PR for this release: https://github.com/firesim/firesim/pull/168
|
||||
|
||||
### Added
|
||||
|
||||
* Supernode support now mainlined
|
||||
* Resolves #11
|
||||
* Includes support for using all 4 host memory channels and connecting them to N targets
|
||||
* FPGA Frequency now configurable in Chisel Config
|
||||
* Printf Synthesis support. See Docs for more info.
|
||||
* Generate elaboration artifacts
|
||||
* Add ccbench workload
|
||||
* Preliminary [GAP Benchmark Suite](https://github.com/sbeamer/gapbs) workload support
|
||||
* PR #223. Adds post_build_hook, dumps `git diff --submodule` into build dir
|
||||
* PR #225. Adds support for building `TARGET_PROJECT` =/= firesim (ex. midasexamples) in the manager
|
||||
* PR #234. Adds support for f1.4xlarge instances
|
||||
* PR #231. fasedtests as a `TARGET_PROJECT` for testing memory models & backing host memory sys.
|
||||
* PR #212. New (alpha) workload generation system "FireMarshal" added
|
||||
|
||||
### Changed
|
||||
|
||||
* PR #218. Bump aws-fpga/FPGA Dev AMI support to 1.4.6 / 1.5.0 respectively.
|
||||
* Resolves #170
|
||||
* According to AWS, this should still work for users on the 1.4.0 AMI
|
||||
* Switch to XDMA from EDMA for DMA transfers. Improves performance ~20% in single-instance cases.
|
||||
* Resolves #51
|
||||
* Only request build-farm instances after successful replace-rtl
|
||||
* Resolves #100
|
||||
* SBT project reworked; FIRRTL provided as an unmanaged dep; target-land annotations pulled into separate project.
|
||||
* Resolves #175
|
||||
* Common DMA RTL factored out into Widget Traits in MIDAS
|
||||
* Boom bumped with RVC Support
|
||||
* Resolves #202
|
||||
* PR #232. Adds separate optimization flags RTL-simulators/driver
|
||||
|
||||
### Fixed
|
||||
|
||||
* Properly generate exit codes in the manager
|
||||
* Resolves #194
|
||||
* Catch build error on infrasetup and log it to file + advise the user to run make command manually
|
||||
* Resolves #69
|
||||
* Fix mem-model bug due to FRFCFS having an under-provisioned functional model
|
||||
* PR #199. Targets with long names can now be killed automatically by firesim
|
||||
* Resolves #56
|
||||
* PR #193. Fedora networking now works in FireSim
|
||||
* Address assignment fixed (gets assigned IP addresses in slot-order on firesim)
|
||||
* PR #204. Fix support for heterogenous rootfs's - each job can have its own rootfs, or no rootfs at all
|
||||
|
||||
### Deprecated
|
||||
|
||||
* None
|
||||
|
||||
### Removed
|
||||
|
||||
* None
|
||||
|
||||
### Security
|
||||
|
||||
* None
|
||||
|
||||
## [1.4.0] - 2018-11-13
|
||||
|
||||
This is a large release. A much more detailed account of everything included is included in the PR: https://github.com/firesim/firesim/pull/114
|
||||
|
|
|
|||
94
README.md
94
README.md
|
|
@ -2,36 +2,69 @@
|
|||
|
||||
|
||||
|
||||
## Contents
|
||||
|
||||
1. [Using FireSim](#using-firesim)
|
||||
2. [What is FireSim?](#what-is-firesim)
|
||||
3. [What can I simulate with FireSim?](#what-can-i-simulate-with-firesim)
|
||||
4. [Need help?](#need-help)
|
||||
5. [Contributing](#contributing)
|
||||
6. [Publications](#publications)
|
||||
|
||||
## Using FireSim
|
||||
|
||||
To get started with using FireSim, see the tutorials on the FireSim documentation
|
||||
site: https://docs.fires.im/
|
||||
site: https://docs.fires.im/.
|
||||
|
||||
Another good overview (in video format) is our tutorial from the Chisel Community Conference on [YouTube](https://www.youtube.com/watch?v=S3OriQnJXYQ).
|
||||
|
||||
## What is FireSim?
|
||||
|
||||
FireSim is an [open-source](https://github.com/firesim/firesim) cycle-accurate,
|
||||
FPGA-accelerated scale-out computer system simulation platform developed in the
|
||||
[Berkeley Architecture Research Group][ucb-bar] in the [Electrical Engineering
|
||||
and Computer Sciences Department][eecs] at the [University of California,
|
||||
Berkeley][berkeley].
|
||||
|
||||
FireSim is capable of simulating from **one to thousands of multi-core compute
|
||||
nodes**, derived directly from **silicon-proven** and **open** target-RTL (e.g. [RISC-V][riscv] [Rocket Chip][rocket-chip] and [BOOM][boom]), with
|
||||
an optional *cycle-accurate network simulation* tying them together. FireSim
|
||||
runs on FPGAs in **public cloud** environments like [AWS EC2
|
||||
F1](https://aws.amazon.com/ec2/instance-types/f1/), removing the high capex
|
||||
traditionally involved in large-scale FPGA-based simulation. FireSim also
|
||||
provides a [Linux distribution](https://github.com/firesim/firesim-software)
|
||||
that is compatible with the systems it simulates and
|
||||
[automates](https://docs.fires.im/en/latest/Advanced-Usage/Workloads/Defining-Custom-Workloads.html)
|
||||
the process of including new workloads into this Linux distribution.
|
||||
|
||||
FireSim is an [open-source](https://github.com/firesim/firesim) cycle-accurate
|
||||
FPGA-accelerated full-system hardware simulation platform that runs on cloud FPGAs (Amazon EC2 F1).
|
||||
FireSim is actively developed in the [Berkeley Architecture Research
|
||||
Group][ucb-bar] in the [Electrical Engineering and Computer Sciences
|
||||
Department][eecs] at the [University of California, Berkeley][berkeley].
|
||||
You can learn more about FireSim in the following places:
|
||||
|
||||
* **FireSim website**: https://fires.im
|
||||
* **FireSim ISCA 2018 Paper**: [Paper PDF](https://sagark.org/assets/pubs/firesim-isca2018.pdf) | [IEEE Xplore](https://ieeexplore.ieee.org/document/8416816) | [ACM DL](https://dl.acm.org/citation.cfm?id=3276543)
|
||||
* **FireSim ISCA 2018 Paper**: [Paper PDF](https://sagark.org/assets/pubs/firesim-isca2018.pdf) | [IEEE Xplore](https://ieeexplore.ieee.org/document/8416816) | [ACM DL](https://dl.acm.org/citation.cfm?id=3276543) | [BibTeX](https://sagark.org/assets/pubs/firesim-isca2018.bib.txt) | Selected as one of IEEE Micro’s “Top Picks from Computer Architecture Conferences, 2018”.
|
||||
* **FireSim documentation**: https://docs.fires.im
|
||||
* **Two-minute lightning talk from ISCA 2018** (FireSim simulating a datacenter): [YouTube](https://www.youtube.com/watch?v=4XwoSe5c8lY)
|
||||
* **Chisel Community Conference Tutorial**: [YouTube](https://www.youtube.com/watch?v=S3OriQnJXYQ)
|
||||
* **Updates/News**: [Changelog](/CHANGELOG.md) | [FireSim Blog](https://fires.im/blog/) | [Twitter](https://twitter.com/firesimproject)
|
||||
|
||||
## What can I simulate with FireSim?
|
||||
|
||||
FireSim can simulate arbitrary hardware designs written in
|
||||
[Chisel](https://chisel.eecs.berkeley.edu). With FireSim, you
|
||||
can write your own RTL (processors, accelerators, etc.) and run it at
|
||||
near-FPGA-prototype speeds on cloud FPGAs, while obtaining cycle-accurate
|
||||
performance results (i.e. matching what you would find if you taped-out
|
||||
a chip). Depending on the hardware design and the simulation scale,
|
||||
FireSim simulations run at **10s to 100s of MHz**. You can also integrate
|
||||
custom software models for components that you don't want/need to write as RTL.
|
||||
|
||||
FireSim was originally developed to simulate datacenters by combining
|
||||
open RTL for RISC-V processors with a custom cycle-accurate network simulation.
|
||||
By default, FireSim provides all the RTL and models necessary
|
||||
to **cycle-exactly** simulate from **one to thousands of multi-core compute
|
||||
nodes**, derived directly from **silicon-proven** and **open** target-RTL
|
||||
([RISC-V][riscv] [Rocket Chip][rocket-chip] and [BOOM][boom]), with an optional
|
||||
**cycle-accurate network simulation** tying them together. FireSim also
|
||||
provides a [Linux distribution](https://github.com/firesim/firesim-software)
|
||||
that is compatible with the RISC-V systems it simulates and
|
||||
[automates](https://docs.fires.im/en/latest/Advanced-Usage/Workloads/Defining-Custom-Workloads.html)
|
||||
the process of including new workloads into this Linux distribution.
|
||||
These simulations run fast
|
||||
enough to interact with Linux on the simulated system at the command line, [like
|
||||
a real
|
||||
computer](https://twitter.com/firesimproject/status/1031267637303508993). Users
|
||||
can even [SSH into simulated systems in
|
||||
FireSim](http://docs.fires.im/en/latest/Advanced-Usage/Miscellaneous-Tips.html#experimental-support-for-sshing-into-simulated-nodes-and-accessing-the-internet-from-within-simulations)
|
||||
and access the Internet from within them.
|
||||
|
||||
Head to the [FireSim Website](https://fires.im) to learn more.
|
||||
|
||||
## Need help?
|
||||
|
||||
|
|
@ -41,23 +74,34 @@ You can learn more about FireSim in the following places:
|
|||
|
||||
## Contributing
|
||||
|
||||
* See CONTRIBUTING.md
|
||||
* See [CONTRIBUTING.md](/CONTRIBUTING.md)
|
||||
|
||||
## ISCA 2018 Paper
|
||||
## Publications
|
||||
|
||||
You can learn more about FireSim in our ISCA 2018 paper, which focuses on
|
||||
FireSim simulations with a globally-cycle-accurate network simulation:
|
||||
### **ISCA 2018**: FireSim: FPGA-Accelerated Cycle-Exact Scale-Out System Simulation in the Public Cloud
|
||||
|
||||
Sagar Karandikar, Howard Mao, Donggyu Kim, David Biancolin, Alon Amid, Dayeol
|
||||
You can learn more about FireSim in our ISCA 2018 paper, which covers the overall FireSim infrastructure and large distributed simulations of networked clusters. This paper was **selected as one of IEEE Micro’s “Top Picks from Computer Architecture Conferences, 2018”.**
|
||||
|
||||
> Sagar Karandikar, Howard Mao, Donggyu Kim, David Biancolin, Alon Amid, Dayeol
|
||||
Lee, Nathan Pemberton, Emmanuel Amaro, Colin Schmidt, Aditya Chopra, Qijing
|
||||
Huang, Kyle Kovacs, Borivoje Nikolic, Randy Katz, Jonathan Bachrach, and Krste
|
||||
Asanović. **FireSim: FPGA-Accelerated Cycle-Exact Scale-Out System Simulation in
|
||||
the Public Cloud**. *In proceedings of the 45th International Symposium
|
||||
on Computer Architecture (ISCA’18)*, Los Angeles, CA, June 2018.
|
||||
|
||||
[Paper PDF](https://sagark.org/assets/pubs/firesim-isca2018.pdf) | [IEEE Xplore](https://ieeexplore.ieee.org/document/8416816) | [ACM DL](https://dl.acm.org/citation.cfm?id=3276543)
|
||||
[Paper PDF](https://sagark.org/assets/pubs/firesim-isca2018.pdf) | [IEEE Xplore](https://ieeexplore.ieee.org/document/8416816) | [ACM DL](https://dl.acm.org/citation.cfm?id=3276543) | [BibTeX](https://sagark.org/assets/pubs/firesim-isca2018.bib.txt)
|
||||
|
||||
|
||||
### **FPGA 2019**: FASED: FPGA-Accelerated Simulation and Evaluation of DRAM
|
||||
|
||||
Our paper from FPGA 2019 details the DRAM model used in FireSim:
|
||||
|
||||
> David Biancolin, Sagar Karandikar, Donggyu Kim, Jack Koenig, Andrew Waterman, Jonathan Bachrach, Krste Asanović, **FASED: FPGA-Accelerated Simulation and Evaluation of DRAM**, In proceedings of the 27th ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, Seaside, CA, February 2018.
|
||||
|
||||
[Paper PDF](http://people.eecs.berkeley.edu/~biancolin/papers/fased-fpga19.pdf)
|
||||
|
||||
You can find other publications, including publications that *use* FireSim on the [FireSim Website](https://fires.im/publications/).
|
||||
|
||||
[ucb-bar]: http://bar.eecs.berkeley.edu
|
||||
[eecs]: https://eecs.berkeley.edu
|
||||
[berkeley]: https://berkeley.edu
|
||||
|
|
|
|||
|
|
@ -92,9 +92,13 @@ echo "export RISCV=$RISCV" > env.sh
|
|||
echo "export PATH=$RISCV/bin:$RDIR/$DTCversion:\$PATH" >> env.sh
|
||||
echo "export LD_LIBRARY_PATH=$RISCV/lib" >> env.sh
|
||||
|
||||
cd "$RDIR/platforms/f1/aws-fpga/sdk/linux_kernel_drivers/edma"
|
||||
cd "$RDIR/platforms/f1/aws-fpga/sdk/linux_kernel_drivers/xdma"
|
||||
make
|
||||
|
||||
# Set up firesim-software
|
||||
cd $RDIR
|
||||
sudo pip3 install -r sw/firesim-software/python-requirements.txt
|
||||
|
||||
# commands to run only on EC2
|
||||
# see if the instance info page exists. if not, we are not on ec2.
|
||||
# this is one of the few methods that works without sudo
|
||||
|
|
|
|||
|
|
@ -14,7 +14,7 @@ rootLogger = logging.getLogger()
|
|||
keyname = 'firesim'
|
||||
|
||||
# this needs to be updated whenever the FPGA Dev AMI changes
|
||||
f1_ami_name = "FPGA Developer AMI - 1.4.0 - pre8-40257ab5-6688-4c95-97d1-e251a40fd1fc-ami-0335b86e84e820e8d.4"
|
||||
f1_ami_name = "FPGA Developer AMI - 1.5.0-40257ab5-6688-4c95-97d1-e251a40fd1fc-ami-06cecb61c79496e0d.4"
|
||||
|
||||
# users are instructed to create these in the setup instructions
|
||||
securitygroupname = 'firesim'
|
||||
|
|
|
|||
|
|
@ -31,24 +31,13 @@ def replace_rtl(conf, buildconfig):
|
|||
rootLogger.info("Running replace-rtl to generate verilog for " + str(buildconfig.get_chisel_triplet()))
|
||||
|
||||
with prefix('cd ' + ddir + '/../'), prefix('source sourceme-f1-manager.sh'), prefix('export CL_DIR={}/../platforms/f1/aws-fpga/{}'.format(ddir, fpgabuilddir)), prefix('cd sim/'), StreamLogger('stdout'), StreamLogger('stderr'):
|
||||
run("""make DESIGN={} TARGET_CONFIG={} PLATFORM_CONFIG={} replace-rtl""".format(
|
||||
buildconfig.DESIGN, buildconfig.TARGET_CONFIG, buildconfig.PLATFORM_CONFIG))
|
||||
run(buildconfig.make_recipe("replace-rtl"))
|
||||
run("""mkdir -p {}/results-build/{}/""".format(ddir, builddir))
|
||||
run("""cp $CL_DIR/design/cl_firesim_generated.sv {}/results-build/{}/cl_firesim_generated.sv""".format(ddir, builddir))
|
||||
|
||||
# build the fpga driver that corresponds with this version of the RTL
|
||||
build_fpga_driver(buildconfig.get_chisel_triplet())
|
||||
|
||||
def build_fpga_driver(triplet):
|
||||
""" Build FPGA driver for running simulation """
|
||||
# TODO there is a duplicate of this in runtools
|
||||
ddir = get_deploy_dir()
|
||||
triplet_pieces = triplet.split("-")
|
||||
design = triplet_pieces[0]
|
||||
target_config = triplet_pieces[1]
|
||||
platform_config = triplet_pieces[2]
|
||||
with prefix('cd ' + ddir + '/../'), prefix('source sourceme-f1-manager.sh'), prefix('cd sim/'), StreamLogger('stdout'), StreamLogger('stderr'):
|
||||
run("""make DESIGN={} TARGET_CONFIG={} PLATFORM_CONFIG={} f1""".format(design, target_config, platform_config))
|
||||
run(buildconfig.make_recipe("f1"))
|
||||
|
||||
@parallel
|
||||
def aws_build(global_build_config, bypass=False):
|
||||
|
|
@ -167,8 +156,8 @@ def aws_build(global_build_config, bypass=False):
|
|||
rootLogger.info("Resulting AFI: " + str(afi))
|
||||
|
||||
rootLogger.info("Waiting for create-fpga-image completion.")
|
||||
|
||||
with lcd("""{}/results-build/{}/""".format(ddir, builddir)), StreamLogger('stdout'), StreamLogger('stderr'):
|
||||
results_build_dir = """{}/results-build/{}/""".format(ddir, builddir)
|
||||
with lcd(results_build_dir), StreamLogger('stdout'), StreamLogger('stderr'):
|
||||
checkstate = "pending"
|
||||
while checkstate == "pending":
|
||||
imagestate = local("""aws ec2 describe-fpga-images --fpga-image-id {} | tee AGFI_INFO""".format(afi), capture=True)
|
||||
|
|
@ -197,7 +186,13 @@ def aws_build(global_build_config, bypass=False):
|
|||
with open(hwdb_entry_file_location + "/" + afiname, "w") as outputfile:
|
||||
outputfile.write(agfi_entry)
|
||||
|
||||
|
||||
if global_build_config.post_build_hook:
|
||||
with StreamLogger('stdout'), StreamLogger('stderr'):
|
||||
localcap = local("""{} {}""".format(global_build_config.post_build_hook,
|
||||
results_build_dir,
|
||||
capture=True))
|
||||
rootLogger.debug("[localhost] " + str(localcap))
|
||||
rootLogger.debug("[localhost] " + str(localcap.stderr))
|
||||
|
||||
rootLogger.info("Build complete! AFI ready. See AGFI_INFO.")
|
||||
rootLogger.info("Terminating the build instance now.")
|
||||
|
|
|
|||
|
|
@ -12,6 +12,7 @@ class BuildConfig:
|
|||
""" This represents a SINGLE build configuration. """
|
||||
def __init__(self, name, buildconfigdict, launch_time):
|
||||
self.name = name
|
||||
self.TARGET_PROJECT = buildconfigdict.get('TARGET_PROJECT')
|
||||
self.DESIGN = buildconfigdict['DESIGN']
|
||||
self.TARGET_CONFIG = buildconfigdict['TARGET_CONFIG']
|
||||
self.PLATFORM_CONFIG = buildconfigdict['PLATFORM_CONFIG']
|
||||
|
|
@ -24,8 +25,7 @@ class BuildConfig:
|
|||
return "BuildConfig obj:\n" + pprint.pformat(vars(self), indent=10)
|
||||
|
||||
def get_chisel_triplet(self):
|
||||
return """{}-{}-{}""".format(self.DESIGN, self.TARGET_CONFIG,
|
||||
self.PLATFORM_CONFIG)
|
||||
return """{}-{}-{}""".format(self.DESIGN, self.TARGET_CONFIG, self.PLATFORM_CONFIG)
|
||||
|
||||
def launch_build_instance(self, build_instance_market,
|
||||
spot_interruption_behavior, spot_max_price):
|
||||
|
|
@ -55,6 +55,15 @@ class BuildConfig:
|
|||
return """{}-{}-{}""".format(self.launch_time,
|
||||
self.get_chisel_triplet(), self.name)
|
||||
|
||||
# Builds up a string for a make invocation using the tuple variables
|
||||
def make_recipe(self, recipe):
|
||||
return """make {} DESIGN={} TARGET_CONFIG={} PLATFORM_CONFIG={} {}""".format(
|
||||
"" if self.TARGET_PROJECT is None else "TARGET_PROJECT=" + self.TARGET_PROJECT,
|
||||
self.DESIGN,
|
||||
self.TARGET_CONFIG,
|
||||
self.PLATFORM_CONFIG,
|
||||
recipe)
|
||||
|
||||
class GlobalBuildConfig:
|
||||
""" Configuration class for builds. This is the "global" configfile, i.e.
|
||||
sample_config_build.ini """
|
||||
|
|
@ -76,6 +85,7 @@ class GlobalBuildConfig:
|
|||
global_build_configfile.get('afibuild', 'spotinterruptionbehavior')
|
||||
self.spot_max_price = \
|
||||
global_build_configfile.get('afibuild', 'spotmaxprice')
|
||||
self.post_build_hook = global_build_configfile.get('afibuild', 'postbuildhook')
|
||||
|
||||
# this is a list of actual builds to run
|
||||
builds_to_run_list = map(lambda x: x[0], global_build_configfile.items('builds'))
|
||||
|
|
|
|||
|
|
@ -105,13 +105,13 @@ def buildafi(globalbuildconf):
|
|||
|
||||
signal.signal(signal.SIGINT, terminate_instances_handler)
|
||||
|
||||
for buildconf in globalbuildconf.get_builds_list():
|
||||
execute(replace_rtl, globalbuildconf, buildconf, hosts=['localhost'])
|
||||
|
||||
# local items (replace_rtl) need to be called in a loop, for each config
|
||||
# remote items will map themselves
|
||||
globalbuildconf.launch_build_instances()
|
||||
|
||||
for buildconf in globalbuildconf.get_builds_list():
|
||||
execute(replace_rtl, globalbuildconf, buildconf, hosts=['localhost'])
|
||||
|
||||
# confirm that build instances have finished booting
|
||||
globalbuildconf.wait_build_instances()
|
||||
|
||||
|
|
@ -139,7 +139,7 @@ def launchrunfarm(runtime_conf):
|
|||
runtime_conf.runfarm.launch_run_farm()
|
||||
|
||||
|
||||
def terminaterunfarm(runtime_conf, terminatesomef1_16, terminatesomef1_2,
|
||||
def terminaterunfarm(runtime_conf, terminatesomef1_16, terminatesomef1_4, terminatesomef1_2,
|
||||
terminatesomem4_16, forceterminate):
|
||||
""" Terminate instances in the runfarm.
|
||||
|
||||
|
|
@ -152,7 +152,7 @@ def terminaterunfarm(runtime_conf, terminatesomef1_16, terminatesomef1_2,
|
|||
that many instances of the specified types and leave all others
|
||||
untouched.
|
||||
"""
|
||||
runtime_conf.terminate_run_farm(terminatesomef1_16, terminatesomef1_2,
|
||||
runtime_conf.terminate_run_farm(terminatesomef1_16, terminatesomef1_4, terminatesomef1_2,
|
||||
terminatesomem4_16, forceterminate)
|
||||
|
||||
def shareagfi(buildconf):
|
||||
|
|
@ -205,6 +205,9 @@ def construct_firesim_argparser():
|
|||
parser.add_argument('-g', '--terminatesomef12', type=int,
|
||||
help='Only used by terminatesome. Terminates this many of the previously launched f1.2xlarges.',
|
||||
default=-1)
|
||||
parser.add_argument('-i', '--terminatesomef14', type=int,
|
||||
help='Only used by terminatesome. Terminates this many of the previously launched f1.4xlarges.',
|
||||
default=-1)
|
||||
parser.add_argument('-m', '--terminatesomem416', type=int,
|
||||
help='Only used by terminatesome. Terminates this many of the previously launched m4.16xlarges.',
|
||||
default=-1)
|
||||
|
|
@ -241,6 +244,7 @@ def main(args):
|
|||
if args.task == 'terminaterunfarm':
|
||||
runtime_conf = RuntimeConfig(args)
|
||||
terminaterunfarm(runtime_conf, args.terminatesomef116,
|
||||
args.terminatesomef14,
|
||||
args.terminatesomef12, args.terminatesomem416,
|
||||
args.forceterminate)
|
||||
|
||||
|
|
@ -298,10 +302,13 @@ if __name__ == '__main__':
|
|||
# it up in "with util.StreamLogger('stdout'), util.StreamLogger('stdin').
|
||||
# unfortunately there's no proper way to do it with fabric
|
||||
|
||||
exitcode = 0
|
||||
try:
|
||||
main(args)
|
||||
except:
|
||||
# log all exceptions that make it this far
|
||||
rootLogger.exception("Fatal error.")
|
||||
exitcode = 1
|
||||
finally:
|
||||
rootLogger.info("""The full log of this run is:\n{basedir}/{fulllog}""".format(basedir=dname, fulllog=full_log_filename))
|
||||
exit(exitcode)
|
||||
|
|
|
|||
|
|
@ -72,6 +72,8 @@ class FireSimLink(object):
|
|||
""" Return True if the user has mapped the two endpoints of this link to
|
||||
separate hosts. This implies a SocketServerPort / SocketClientPort will be used
|
||||
to implement the Link. If False, use a sharedmem port to implement the link. """
|
||||
if type(self.get_downlink_side()) == FireSimDummyServerNode:
|
||||
return False
|
||||
return self.get_uplink_side().host_instance != self.get_downlink_side().host_instance
|
||||
|
||||
def get_global_link_id(self):
|
||||
|
|
@ -216,20 +218,21 @@ class FireSimServerNode(FireSimNode):
|
|||
rootLogger.debug("[localhost] " + str(localcap.stderr))
|
||||
|
||||
# mount rootfs, copy files from it back to local system
|
||||
mountpoint = """/home/centos/sim_slot_{}/mountpoint""".format(simserverindex)
|
||||
with StreamLogger('stdout'), StreamLogger('stderr'):
|
||||
run("""sudo mkdir -p {}""".format(mountpoint))
|
||||
run("""sudo mount /home/centos/sim_slot_{}/{} {}""".format(simserverindex, self.get_rootfs_name(), mountpoint))
|
||||
run("""sudo chmod -Rf 777 {}""".format(mountpoint))
|
||||
if self.get_rootfs_name() is not None:
|
||||
mountpoint = """/home/centos/sim_slot_{}/mountpoint""".format(simserverindex)
|
||||
with StreamLogger('stdout'), StreamLogger('stderr'):
|
||||
run("""sudo mkdir -p {}""".format(mountpoint))
|
||||
run("""sudo mount /home/centos/sim_slot_{}/{} {}""".format(simserverindex, self.get_rootfs_name(), mountpoint))
|
||||
run("""sudo chmod -Rf 777 {}""".format(mountpoint))
|
||||
|
||||
## copy back files from inside the rootfs
|
||||
with warn_only(), StreamLogger('stdout'), StreamLogger('stderr'):
|
||||
for outputfile in jobinfo.outputs:
|
||||
get(remote_path=mountpoint + outputfile, local_path=job_dir)
|
||||
## copy back files from inside the rootfs
|
||||
with warn_only(), StreamLogger('stdout'), StreamLogger('stderr'):
|
||||
for outputfile in jobinfo.outputs:
|
||||
get(remote_path=mountpoint + outputfile, local_path=job_dir)
|
||||
|
||||
## unmount
|
||||
with StreamLogger('stdout'), StreamLogger('stderr'):
|
||||
run("""sudo umount {}""".format(mountpoint))
|
||||
## unmount
|
||||
with StreamLogger('stdout'), StreamLogger('stderr'):
|
||||
run("""sudo umount {}""".format(mountpoint))
|
||||
|
||||
## copy output files generated by the simulator that live on the host:
|
||||
## e.g. uartlog, memory_stats.csv, etc
|
||||
|
|
@ -248,13 +251,15 @@ class FireSimServerNode(FireSimNode):
|
|||
""" Return local paths of all stuff needed to run this simulation as
|
||||
an array. """
|
||||
all_paths = []
|
||||
# todo handle none case
|
||||
all_paths.append(self.get_job().rootfs_path())
|
||||
all_paths.append(self.get_job().bootbinary_path())
|
||||
|
||||
all_paths.append(self.server_hardware_config.get_local_driver_path())
|
||||
all_paths.append(self.server_hardware_config.get_local_runtime_conf_path())
|
||||
all_paths.append(self.server_hardware_config.get_local_assert_def_path())
|
||||
if self.get_job().rootfs_path() is not None:
|
||||
all_paths.append([self.get_job().rootfs_path(), ''])
|
||||
|
||||
all_paths.append([self.get_job().bootbinary_path(), ''])
|
||||
|
||||
all_paths.append([self.server_hardware_config.get_local_driver_path(), ''])
|
||||
all_paths.append([self.server_hardware_config.get_local_runtime_conf_path(), ''])
|
||||
all_paths.append([self.server_hardware_config.get_local_assert_def_path(), ''])
|
||||
return all_paths
|
||||
|
||||
def get_agfi(self):
|
||||
|
|
@ -272,13 +277,156 @@ class FireSimServerNode(FireSimNode):
|
|||
def get_job_name(self):
|
||||
return self.job.jobname
|
||||
|
||||
def get_rootfs_name(self):
|
||||
return self.get_job().rootfs_path().split("/")[-1]
|
||||
def get_rootfs_name(self, dummyindex=0):
|
||||
if self.get_job().rootfs_path() is None:
|
||||
return None
|
||||
elif dummyindex:
|
||||
return self.get_job().rootfs_path().split("/")[-1] + "-" + str(dummyindex)
|
||||
else:
|
||||
return self.get_job().rootfs_path().split("/")[-1]
|
||||
|
||||
def get_bootbin_name(self):
|
||||
def get_bootbin_name(self, dummyindex=0):
|
||||
if dummyindex:
|
||||
return self.get_job().bootbinary_path().split("/")[-1] + "-" + str(dummyindex)
|
||||
return self.get_job().bootbinary_path().split("/")[-1]
|
||||
|
||||
|
||||
class FireSimSuperNodeServerNode(FireSimServerNode):
|
||||
""" This is the main server node for supernode mode. This knows how to
|
||||
call out to dummy server nodes to get all the info to launch the one
|
||||
command line to run the FPGA sim that has N > 1 sims on one fpga."""
|
||||
|
||||
def supernode_get_num_siblings(self):
|
||||
siblings = 0
|
||||
count = False
|
||||
for index, servernode in enumerate(map( lambda x : x.get_downlink_side(), self.uplinks[0].get_uplink_side().downlinks)):
|
||||
if count:
|
||||
if isinstance(servernode, FireSimDummyServerNode):
|
||||
count += 1
|
||||
else:
|
||||
return count
|
||||
elif self == servernode:
|
||||
count = True
|
||||
return count
|
||||
|
||||
def supernode_get_sibling(self, siblingindex):
|
||||
""" return the sibling for supernode mode.
|
||||
siblingindex = 1 -> next sibling, 2 = second, 3 = last one."""
|
||||
for index, servernode in enumerate(map( lambda x : x.get_downlink_side(), self.uplinks[0].get_uplink_side().downlinks)):
|
||||
if self == servernode:
|
||||
return self.uplinks[0].get_uplink_side().downlinks[index+siblingindex].get_downlink_side()
|
||||
|
||||
def supernode_get_sibling_mac_address(self, siblingindex):
|
||||
""" return the sibling's mac address for supernode mode.
|
||||
siblingindex = 1 -> next sibling, 2 = second, 3 = last one."""
|
||||
return self.supernode_get_sibling(siblingindex).get_mac_address()
|
||||
|
||||
def supernode_get_sibling_rootfs(self, siblingindex):
|
||||
""" return the sibling's rootfs for supernode mode.
|
||||
siblingindex = 1 -> next sibling, 2 = second, 3 = last one."""
|
||||
return self.supernode_get_sibling(siblingindex).get_rootfs_name(siblingindex)
|
||||
|
||||
def supernode_get_sibling_bootbin(self, siblingindex):
|
||||
""" return the sibling's rootfs for supernode mode.
|
||||
siblingindex = 1 -> next sibling, 2 = second, 3 = last one."""
|
||||
return self.supernode_get_sibling(siblingindex).get_bootbin_name(siblingindex)
|
||||
|
||||
def supernode_get_sibling_rootfs_path(self, siblingindex):
|
||||
return self.supernode_get_sibling(siblingindex).get_job().rootfs_path()
|
||||
|
||||
def supernode_get_sibling_bootbinary_path(self, siblingindex):
|
||||
return self.supernode_get_sibling(siblingindex).get_job().bootbinary_path()
|
||||
|
||||
def supernode_get_sibling_link_latency(self, siblingindex):
|
||||
return self.supernode_get_sibling(siblingindex).server_link_latency
|
||||
|
||||
def supernode_get_sibling_bw_max(self, siblingindex):
|
||||
return self.supernode_get_sibling(siblingindex).server_bw_max
|
||||
|
||||
def supernode_get_sibling_shmemportname(self, siblingindex):
|
||||
return self.supernode_get_sibling(siblingindex).uplinks[0].get_global_link_id()
|
||||
|
||||
def get_sim_start_command(self, slotno):
|
||||
""" return the command to start the simulation. assumes it will be
|
||||
called in a directory where its required_files are already located.
|
||||
|
||||
Currently hardcoded to 4 nodes.
|
||||
"""
|
||||
|
||||
num_siblings = self.supernode_get_num_siblings()
|
||||
|
||||
all_macs = [self.get_mac_address()] + [self.supernode_get_sibling_mac_address(x) for x in range(1, num_siblings)]
|
||||
all_rootfses = [self.get_rootfs_name()] + [self.supernode_get_sibling_rootfs(x) for x in range(1, num_siblings)]
|
||||
all_bootbins = [self.get_bootbin_name()] + [self.supernode_get_sibling_bootbin(x) for x in range(1, num_siblings)]
|
||||
all_linklatencies = [self.server_link_latency] + [self.supernode_get_sibling_link_latency(x) for x in range(1, num_siblings)]
|
||||
all_maxbws = [self.server_bw_max] + [self.supernode_get_sibling_bw_max(x) for x in range(1, num_siblings)]
|
||||
|
||||
all_shmemportnames = ["default" for x in range(num_siblings)]
|
||||
if self.uplinks:
|
||||
all_shmemportnames = [self.uplinks[0].get_global_link_id()] + [self.supernode_get_sibling_shmemportname(x) for x in range(1, num_siblings)]
|
||||
|
||||
return self.server_hardware_config.get_supernode_boot_simulation_command(
|
||||
slotno, all_macs, all_rootfses, all_linklatencies, all_maxbws,
|
||||
self.server_profile_interval, all_bootbins, self.trace_enable,
|
||||
self.trace_start, self.trace_end, all_shmemportnames)
|
||||
|
||||
def get_required_files_local_paths(self):
|
||||
""" Return local paths of all stuff needed to run this simulation as
|
||||
an array. """
|
||||
|
||||
def get_path_trailing(filepath):
|
||||
return filepath.split("/")[-1]
|
||||
def local_and_remote(filepath, index):
|
||||
return [filepath, get_path_trailing(filepath) + str(index)]
|
||||
|
||||
all_paths = []
|
||||
if self.get_job().rootfs_path() is not None:
|
||||
all_paths.append([self.get_job().rootfs_path(),
|
||||
self.get_rootfs_name()])
|
||||
|
||||
num_siblings = self.supernode_get_num_siblings()
|
||||
|
||||
for x in range(1, num_siblings):
|
||||
sibling_rootfs_path = self.supernode_get_sibling_rootfs_path(x)
|
||||
if sibling_rootfs_path is not None:
|
||||
all_paths.append([sibling_rootfs_path,
|
||||
self.supernode_get_sibling_rootfs(x)])
|
||||
|
||||
all_paths.append([self.get_job().bootbinary_path(),
|
||||
self.get_bootbin_name()])
|
||||
|
||||
for x in range(1, num_siblings):
|
||||
all_paths.append([self.supernode_get_sibling_bootbinary_path(x),
|
||||
self.supernode_get_sibling_bootbin(x)])
|
||||
|
||||
all_paths.append([self.server_hardware_config.get_local_driver_path(), ''])
|
||||
all_paths.append([self.server_hardware_config.get_local_runtime_conf_path(), ''])
|
||||
all_paths.append([self.server_hardware_config.get_local_assert_def_path(), ''])
|
||||
return all_paths
|
||||
|
||||
def get_rootfs_name(self, dummyindex=0):
|
||||
if self.get_job().rootfs_path() is None:
|
||||
return None
|
||||
elif dummyindex:
|
||||
return self.get_job().rootfs_path().split("/")[-1] + "-" + str(dummyindex)
|
||||
else:
|
||||
return self.get_job().rootfs_path().split("/")[-1]
|
||||
|
||||
def get_bootbin_name(self, dummyindex=0):
|
||||
if dummyindex:
|
||||
return self.get_job().bootbinary_path().split("/")[-1] + "-" + str(dummyindex)
|
||||
return self.get_job().bootbinary_path().split("/")[-1]
|
||||
|
||||
|
||||
class FireSimDummyServerNode(FireSimServerNode):
|
||||
""" This is a dummy server node for supernode mode. """
|
||||
def __init__(self, server_hardware_config=None, server_link_latency=None,
|
||||
server_bw_max=None):
|
||||
super(FireSimDummyServerNode, self).__init__(server_hardware_config,
|
||||
server_link_latency,
|
||||
server_bw_max)
|
||||
|
||||
|
||||
class FireSimSwitchNode(FireSimNode):
|
||||
""" This is a simulated switch instance in FireSim.
|
||||
|
||||
|
|
|
|||
|
|
@ -162,26 +162,34 @@ class FireSimTopologyWithPasses:
|
|||
return
|
||||
assert serverind == len(servers), "ERR: all servers were not assigned to a host."
|
||||
|
||||
|
||||
def pass_simple_networked_host_node_mapping(self):
|
||||
""" A very simple host mapping strategy. """
|
||||
switches = self.firesimtopol.get_dfs_order_switches()
|
||||
f1_2s_used = 0
|
||||
f1_4s_used = 0
|
||||
f1_16s_used = 0
|
||||
m4_16s_used = 0
|
||||
|
||||
for switch in switches:
|
||||
downlinknodes = map(lambda x: x.get_downlink_side(), switch.downlinks)
|
||||
# Filter out FireSimDummyServerNodes for actually deploying.
|
||||
# Infrastructure after this point will automatically look at the
|
||||
# FireSimDummyServerNodes if a FireSimSuperNodeServerNode is used
|
||||
downlinknodes = map(lambda x: x.get_downlink_side(), [downlink for downlink in switch.downlinks if not isinstance(downlink.get_downlink_side(), FireSimDummyServerNode)])
|
||||
if all([isinstance(x, FireSimSwitchNode) for x in downlinknodes]):
|
||||
# all downlinks are switches
|
||||
self.run_farm.m4_16s[m4_16s_used].add_switch(switch)
|
||||
m4_16s_used += 1
|
||||
elif all([isinstance(x, FireSimServerNode) for x in downlinknodes]):
|
||||
# all downlinks are simulations
|
||||
if (len(switch.downlinks) == 1) and (f1_2s_used < len(self.run_farm.f1_2s)):
|
||||
if (len(downlinknodes) == 1) and (f1_2s_used < len(self.run_farm.f1_2s)):
|
||||
self.run_farm.f1_2s[f1_2s_used].add_switch(switch)
|
||||
self.run_farm.f1_2s[f1_2s_used].add_simulation(downlinknodes[0])
|
||||
f1_2s_used += 1
|
||||
elif (len(downlinknodes) == 2) and (f1_4s_used < len(self.run_farm.f1_4s)):
|
||||
self.run_farm.f1_4s[f1_4s_used].add_switch(switch)
|
||||
for server in downlinknodes:
|
||||
self.run_farm.f1_4s[f1_4s_used].add_simulation(server)
|
||||
f1_4s_used += 1
|
||||
else:
|
||||
self.run_farm.f1_16s[f1_16s_used].add_switch(switch)
|
||||
for server in downlinknodes:
|
||||
|
|
@ -347,12 +355,20 @@ class FireSimTopologyWithPasses:
|
|||
execute(instance_liveness, hosts=all_runfarm_ips)
|
||||
execute(infrasetup_node_wrapper, self.run_farm, hosts=all_runfarm_ips)
|
||||
|
||||
def boot_simulation_passes(self, use_mock_instances_for_testing):
|
||||
""" Passes that setup for boot and boot the simulation. """
|
||||
if use_mock_instances_for_testing:
|
||||
self.run_farm.bind_mock_instances_to_objects()
|
||||
else:
|
||||
self.run_farm.bind_real_instances_to_objects()
|
||||
def boot_simulation_passes(self, use_mock_instances_for_testing, skip_instance_binding=False):
|
||||
""" Passes that setup for boot and boot the simulation.
|
||||
skip instance binding lets users not call the binding pass on the run_farm
|
||||
again, e.g. if this was called by runworkload (because runworkload calls
|
||||
boot_simulation_passes internally)
|
||||
TODO: the reason we need this is that somehow we're getting
|
||||
garbage results if the AWS EC2 API gets called twice by accident
|
||||
(e.g. incorrect private IPs)
|
||||
"""
|
||||
if not skip_instance_binding:
|
||||
if use_mock_instances_for_testing:
|
||||
self.run_farm.bind_mock_instances_to_objects()
|
||||
else:
|
||||
self.run_farm.bind_real_instances_to_objects()
|
||||
|
||||
@parallel
|
||||
def boot_switch_wrapper(runfarm):
|
||||
|
|
@ -426,7 +442,7 @@ class FireSimTopologyWithPasses:
|
|||
rootLogger.debug("[localhost] " + str(localcap.stderr))
|
||||
|
||||
# boot up as usual
|
||||
self.boot_simulation_passes(False)
|
||||
self.boot_simulation_passes(False, skip_instance_binding=True)
|
||||
|
||||
@parallel
|
||||
def monitor_jobs_wrapper(runfarm, completed_jobs, teardown, terminateoncompletion, job_results_dir):
|
||||
|
|
|
|||
|
|
@ -7,9 +7,17 @@ from awstools.awstools import *
|
|||
from fabric.api import *
|
||||
from fabric.contrib.project import rsync_project
|
||||
from util.streamlogger import StreamLogger
|
||||
import time
|
||||
|
||||
rootLogger = logging.getLogger()
|
||||
|
||||
def remote_kmsg(message):
|
||||
""" This will let you write whatever is passed as message into the kernel
|
||||
log of the remote machine. Useful for figuring what the manager is doing
|
||||
w.r.t output from kernel stuff on the remote node. """
|
||||
commd = """echo '{}' | sudo tee /dev/kmsg""".format(message)
|
||||
run(commd, shell=True)
|
||||
|
||||
class MockBoto3Instance:
|
||||
""" This is used for testing without actually launching instances. """
|
||||
|
||||
|
|
@ -94,6 +102,16 @@ class F1_16(F1_Instance):
|
|||
self.instance_id = F1_16.instance_counter
|
||||
F1_16.instance_counter += 1
|
||||
|
||||
class F1_4(F1_Instance):
|
||||
instance_counter = 0
|
||||
FPGA_SLOTS = 2
|
||||
|
||||
def __init__(self):
|
||||
super(F1_4, self).__init__()
|
||||
self.fpga_slots = [None for x in range(self.FPGA_SLOTS)]
|
||||
self.instance_id = F1_4.instance_counter
|
||||
F1_4.instance_counter += 1
|
||||
|
||||
class F1_2(F1_Instance):
|
||||
instance_counter = 0
|
||||
FPGA_SLOTS = 1
|
||||
|
|
@ -119,10 +137,11 @@ class RunFarm:
|
|||
This way, you can assign "instances" to simulations first, and then assign
|
||||
the real instance ids to the instance objects managed here."""
|
||||
|
||||
def __init__(self, num_f1_16, num_f1_2, num_m4_16, runfarmtag,
|
||||
def __init__(self, num_f1_16, num_f1_4, num_f1_2, num_m4_16, runfarmtag,
|
||||
run_instance_market, spot_interruption_behavior,
|
||||
spot_max_price):
|
||||
self.f1_16s = [F1_16() for x in range(num_f1_16)]
|
||||
self.f1_4s = [F1_4() for x in range(num_f1_4)]
|
||||
self.f1_2s = [F1_2() for x in range(num_f1_2)]
|
||||
self.m4_16s = [M4_16() for x in range(num_m4_16)]
|
||||
|
||||
|
|
@ -136,6 +155,9 @@ class RunFarm:
|
|||
for index in range(len(self.f1_16s)):
|
||||
self.f1_16s[index].assign_boto3_instance_object(MockBoto3Instance())
|
||||
|
||||
for index in range(len(self.f1_4s)):
|
||||
self.f1_4s[index].assign_boto3_instance_object(MockBoto3Instance())
|
||||
|
||||
for index in range(len(self.f1_2s)):
|
||||
self.f1_2s[index].assign_boto3_instance_object(MockBoto3Instance())
|
||||
|
||||
|
|
@ -149,6 +171,8 @@ class RunFarm:
|
|||
# we always sort by private IP when handling instances
|
||||
available_f1_16_instances = instances_sorted_by_avail_ip(get_instances_by_tag_type(
|
||||
self.runfarmtag, 'f1.16xlarge'))
|
||||
available_f1_4_instances = instances_sorted_by_avail_ip(get_instances_by_tag_type(
|
||||
self.runfarmtag, 'f1.4xlarge'))
|
||||
available_m4_16_instances = instances_sorted_by_avail_ip(get_instances_by_tag_type(
|
||||
self.runfarmtag, 'm4.16xlarge'))
|
||||
available_f1_2_instances = instances_sorted_by_avail_ip(get_instances_by_tag_type(
|
||||
|
|
@ -158,19 +182,26 @@ class RunFarm:
|
|||
# confirm that we have the correct number of instances
|
||||
if not (len(available_f1_16_instances) >= len(self.f1_16s)):
|
||||
rootLogger.warning(message.format("f1.16xlarges"))
|
||||
if not (len(available_f1_4_instances) >= len(self.f1_4s)):
|
||||
rootLogger.warning(message.format("f1.4xlarges"))
|
||||
if not (len(available_f1_2_instances) >= len(self.f1_2s)):
|
||||
rootLogger.warning(message.format("f1.2xlarges"))
|
||||
if not (len(available_f1_16_instances) >= len(self.f1_16s)):
|
||||
if not (len(available_m4_16_instances) >= len(self.m4_16s)):
|
||||
rootLogger.warning(message.format("m4.16xlarges"))
|
||||
|
||||
#self.f1_16x_ips = get_private_ips_for_instances(f1_16_instances)
|
||||
#self.m4_16x_ips = get_private_ips_for_instances(m4_16_instances)
|
||||
#self.f1_2x_ips = get_private_ips_for_instances(f1_2_instances)
|
||||
ipmessage = """Using {} instances with IPs:\n{}"""
|
||||
rootLogger.debug(ipmessage.format("f1.16xlarge", str(get_private_ips_for_instances(available_f1_16_instances))))
|
||||
rootLogger.debug(ipmessage.format("f1.4xlarge", str(get_private_ips_for_instances(available_f1_4_instances))))
|
||||
rootLogger.debug(ipmessage.format("f1.2xlarge", str(get_private_ips_for_instances(available_f1_2_instances))))
|
||||
rootLogger.debug(ipmessage.format("m4.16xlarge", str(get_private_ips_for_instances(available_m4_16_instances))))
|
||||
|
||||
# assign boto3 instance objects to our instance objects
|
||||
for index, instance in enumerate(available_f1_16_instances):
|
||||
self.f1_16s[index].assign_boto3_instance_object(instance)
|
||||
|
||||
for index, instance in enumerate(available_f1_4_instances):
|
||||
self.f1_4s[index].assign_boto3_instance_object(instance)
|
||||
|
||||
for index, instance in enumerate(available_m4_16_instances):
|
||||
self.m4_16s[index].assign_boto3_instance_object(instance)
|
||||
|
||||
|
|
@ -186,6 +217,7 @@ class RunFarm:
|
|||
spotmaxprice = self.spot_max_price
|
||||
|
||||
num_f1_16xlarges = len(self.f1_16s)
|
||||
num_f1_4xlarges = len(self.f1_4s)
|
||||
num_f1_2xlarges = len(self.f1_2s)
|
||||
num_m4_16xlarges = len(self.m4_16s)
|
||||
|
||||
|
|
@ -193,6 +225,9 @@ class RunFarm:
|
|||
f1_16s = launch_run_instances('f1.16xlarge', num_f1_16xlarges, runfarmtag,
|
||||
runinstancemarket, spotinterruptionbehavior,
|
||||
spotmaxprice)
|
||||
f1_4s = launch_run_instances('f1.4xlarge', num_f1_4xlarges, runfarmtag,
|
||||
runinstancemarket, spotinterruptionbehavior,
|
||||
spotmaxprice)
|
||||
m4_16s = launch_run_instances('m4.16xlarge', num_m4_16xlarges, runfarmtag,
|
||||
runinstancemarket, spotinterruptionbehavior,
|
||||
spotmaxprice)
|
||||
|
|
@ -204,11 +239,12 @@ class RunFarm:
|
|||
# TODO: maybe we shouldn't do this, but just let infrasetup block. That
|
||||
# way we get builds out of the way while waiting for instances to launch
|
||||
wait_on_instance_launches(f1_16s, 'f1.16xlarges')
|
||||
wait_on_instance_launches(f1_4s, 'f1.4xlarges')
|
||||
wait_on_instance_launches(m4_16s, 'm4.16xlarges')
|
||||
wait_on_instance_launches(f1_2s, 'f1.2xlarges')
|
||||
|
||||
|
||||
def terminate_run_farm(self, terminatesomef1_16, terminatesomef1_2,
|
||||
def terminate_run_farm(self, terminatesomef1_16, terminatesomef1_4, terminatesomef1_2,
|
||||
terminatesomem4_16, forceterminate):
|
||||
runfarmtag = self.runfarmtag
|
||||
|
||||
|
|
@ -216,20 +252,24 @@ class RunFarm:
|
|||
# terminating some, to try to get intra-availability-zone locality
|
||||
f1_16_instances = instances_sorted_by_avail_ip(
|
||||
get_instances_by_tag_type(runfarmtag, 'f1.16xlarge'))
|
||||
f1_4_instances = instances_sorted_by_avail_ip(
|
||||
get_instances_by_tag_type(runfarmtag, 'f1.4xlarge'))
|
||||
m4_16_instances = instances_sorted_by_avail_ip(
|
||||
get_instances_by_tag_type(runfarmtag, 'm4.16xlarge'))
|
||||
f1_2_instances = instances_sorted_by_avail_ip(
|
||||
get_instances_by_tag_type(runfarmtag, 'f1.2xlarge'))
|
||||
|
||||
f1_16_instance_ids = get_instance_ids_for_instances(f1_16_instances)
|
||||
f1_4_instance_ids = get_instance_ids_for_instances(f1_4_instances)
|
||||
m4_16_instance_ids = get_instance_ids_for_instances(m4_16_instances)
|
||||
f1_2_instance_ids = get_instance_ids_for_instances(f1_2_instances)
|
||||
|
||||
argsupplied_f116 = terminatesomef1_16 != -1
|
||||
argsupplied_f14 = terminatesomef1_4 != -1
|
||||
argsupplied_f12 = terminatesomef1_2 != -1
|
||||
argsupplied_m416 = terminatesomem4_16 != -1
|
||||
|
||||
if argsupplied_f116 or argsupplied_f12 or argsupplied_m416:
|
||||
if argsupplied_f116 or argsupplied_f14 or argsupplied_f12 or argsupplied_m416:
|
||||
# In this mode, only terminate instances that are specifically supplied.
|
||||
if argsupplied_f116 and terminatesomef1_16 != 0:
|
||||
# grab the last N instances to terminate
|
||||
|
|
@ -237,6 +277,12 @@ class RunFarm:
|
|||
else:
|
||||
f1_16_instance_ids = []
|
||||
|
||||
if argsupplied_f14 and terminatesomef1_4 != 0:
|
||||
# grab the last N instances to terminate
|
||||
f1_4_instance_ids = f1_4_instance_ids[-terminatesomef1_4:]
|
||||
else:
|
||||
f1_4_instance_ids = []
|
||||
|
||||
if argsupplied_f12 and terminatesomef1_2 != 0:
|
||||
# grab the last N instances to terminate
|
||||
f1_2_instance_ids = f1_2_instance_ids[-terminatesomef1_2:]
|
||||
|
|
@ -252,6 +298,8 @@ class RunFarm:
|
|||
rootLogger.critical("IMPORTANT!: This will terminate the following instances:")
|
||||
rootLogger.critical("f1.16xlarges")
|
||||
rootLogger.critical(f1_16_instance_ids)
|
||||
rootLogger.critical("f1.4xlarges")
|
||||
rootLogger.critical(f1_4_instance_ids)
|
||||
rootLogger.critical("m4.16xlarges")
|
||||
rootLogger.critical(m4_16_instance_ids)
|
||||
rootLogger.critical("f1.2xlarges")
|
||||
|
|
@ -266,6 +314,8 @@ class RunFarm:
|
|||
if userconfirm == "yes":
|
||||
if len(f1_16_instance_ids) != 0:
|
||||
terminate_instances(f1_16_instance_ids, False)
|
||||
if len(f1_4_instance_ids) != 0:
|
||||
terminate_instances(f1_4_instance_ids, False)
|
||||
if len(m4_16_instance_ids) != 0:
|
||||
terminate_instances(m4_16_instance_ids, False)
|
||||
if len(f1_2_instance_ids) != 0:
|
||||
|
|
@ -277,7 +327,7 @@ class RunFarm:
|
|||
def get_all_host_nodes(self):
|
||||
""" Get objects for all host nodes in the run farm that are bound to
|
||||
a real instance. """
|
||||
allinsts = self.f1_16s + self.f1_2s + self.m4_16s
|
||||
allinsts = self.f1_16s + self.f1_2s + self.f1_4s + self.m4_16s
|
||||
return [inst for inst in allinsts if inst.boto3_instance_object is not None]
|
||||
|
||||
def lookup_by_ip_addr(self, ipaddr):
|
||||
|
|
@ -307,7 +357,7 @@ class InstanceDeployManager:
|
|||
# TODO: we checkout a specific version of aws-fpga here, in case upstream
|
||||
# master is bumped. But now we have to remember to change AWS_FPGA_FIRESIM_UPSTREAM_VERSION
|
||||
# when we bump our stuff. Need a better way to do this.
|
||||
AWS_FPGA_FIRESIM_UPSTREAM_VERSION = "2fdf23ffad944cb94f98d09eed0f34c220c522fe"
|
||||
AWS_FPGA_FIRESIM_UPSTREAM_VERSION = "e5b68dd8d432c746f7094b54abf35334bc51b9d1"
|
||||
self.instance_logger("""Installing AWS FPGA SDK on remote nodes. Upstream hash: {}""".format(AWS_FPGA_FIRESIM_UPSTREAM_VERSION))
|
||||
with warn_only(), StreamLogger('stdout'), StreamLogger('stderr'):
|
||||
run('git clone https://github.com/aws/aws-fpga')
|
||||
|
|
@ -315,54 +365,97 @@ class InstanceDeployManager:
|
|||
with cd('/home/centos/aws-fpga'), StreamLogger('stdout'), StreamLogger('stderr'):
|
||||
run('source sdk_setup.sh')
|
||||
|
||||
def fpga_node_edma(self):
|
||||
""" Copy EDMA infra to remote node. This assumes that the driver was
|
||||
def fpga_node_xdma(self):
|
||||
""" Copy XDMA infra to remote node. This assumes that the driver was
|
||||
already built and that a binary exists in the directory on this machine
|
||||
"""
|
||||
self.instance_logger("""Copying AWS FPGA EDMA driver to remote node.""")
|
||||
self.instance_logger("""Copying AWS FPGA XDMA driver to remote node.""")
|
||||
with StreamLogger('stdout'), StreamLogger('stderr'):
|
||||
run('mkdir -p /home/centos/edma/')
|
||||
run('mkdir -p /home/centos/xdma/')
|
||||
put('../platforms/f1/aws-fpga/sdk/linux_kernel_drivers',
|
||||
'/home/centos/edma/', mirror_local_mode=True)
|
||||
with cd('/home/centos/edma/linux_kernel_drivers/edma/'):
|
||||
'/home/centos/xdma/', mirror_local_mode=True)
|
||||
with cd('/home/centos/xdma/linux_kernel_drivers/xdma/'):
|
||||
run('make clean')
|
||||
run('make')
|
||||
|
||||
def unload_edma(self):
|
||||
self.instance_logger("Unloading EDMA Driver Kernel Module.")
|
||||
def unload_xdma(self):
|
||||
self.instance_logger("Unloading XDMA/EDMA/XOCL Driver Kernel Module.")
|
||||
|
||||
with warn_only(), StreamLogger('stdout'), StreamLogger('stderr'):
|
||||
run('sudo rmmod edma-drv')
|
||||
# fpga mgmt tools seem to force load xocl after a flash now...
|
||||
# so we just remove everything for good measure:
|
||||
remote_kmsg("removing_xdma_start")
|
||||
run('sudo rmmod xocl')
|
||||
run('sudo rmmod xdma')
|
||||
run('sudo rmmod edma')
|
||||
remote_kmsg("removing_xdma_end")
|
||||
|
||||
#self.instance_logger("Waiting 10 seconds after removing kernel modules (esp. xocl).")
|
||||
#time.sleep(10)
|
||||
|
||||
def clear_fpgas(self):
|
||||
# we always clear ALL fpga slots
|
||||
for slotno in range(self.parentnode.get_num_fpga_slots_max()):
|
||||
self.instance_logger("""Clearing FPGA Slot {}.""".format(slotno))
|
||||
with StreamLogger('stdout'), StreamLogger('stderr'):
|
||||
remote_kmsg("""about_to_clear_fpga{}""".format(slotno))
|
||||
run("""sudo fpga-clear-local-image -S {} -A""".format(slotno))
|
||||
remote_kmsg("""done_clearing_fpga{}""".format(slotno))
|
||||
|
||||
for slotno in range(self.parentnode.get_num_fpga_slots_max()):
|
||||
self.instance_logger("""Checking for Cleared FPGA Slot {}.""".format(slotno))
|
||||
with StreamLogger('stdout'), StreamLogger('stderr'):
|
||||
remote_kmsg("""about_to_check_clear_fpga{}""".format(slotno))
|
||||
run("""until sudo fpga-describe-local-image -S {} -R -H | grep -q "cleared"; do sleep 1; done""".format(slotno))
|
||||
remote_kmsg("""done_checking_clear_fpga{}""".format(slotno))
|
||||
|
||||
|
||||
def flash_fpgas(self):
|
||||
dummyagfi = None
|
||||
for firesimservernode, slotno in zip(self.parentnode.fpga_slots, range(self.parentnode.get_num_fpga_slots_consumed())):
|
||||
if firesimservernode is not None:
|
||||
agfi = firesimservernode.get_agfi()
|
||||
dummyagfi = agfi
|
||||
self.instance_logger("""Flashing FPGA Slot: {} with agfi: {}.""".format(slotno, agfi))
|
||||
with StreamLogger('stdout'), StreamLogger('stderr'):
|
||||
run("""sudo fpga-load-local-image -S {} -I {} -A""".format(
|
||||
slotno, agfi))
|
||||
|
||||
# We only do this because XDMA hangs if some of the FPGAs on the instance
|
||||
# are left in the cleared state. So, if you're only using some of the
|
||||
# FPGAs on an instance, we flash the rest with one of your images
|
||||
# anyway. Since the only interaction we have with an FPGA right now
|
||||
# is over PCIe where the software component is mastering, this can't
|
||||
# break anything.
|
||||
for slotno in range(self.parentnode.get_num_fpga_slots_consumed(), self.parentnode.get_num_fpga_slots_max()):
|
||||
self.instance_logger("""Flashing FPGA Slot: {} with dummy agfi: {}.""".format(slotno, dummyagfi))
|
||||
with StreamLogger('stdout'), StreamLogger('stderr'):
|
||||
run("""sudo fpga-load-local-image -S {} -I {} -A""".format(
|
||||
slotno, dummyagfi))
|
||||
|
||||
for firesimservernode, slotno in zip(self.parentnode.fpga_slots, range(self.parentnode.get_num_fpga_slots_consumed())):
|
||||
if firesimservernode is not None:
|
||||
self.instance_logger("""Checking for Flashed FPGA Slot: {} with agfi: {}.""".format(slotno, agfi))
|
||||
with StreamLogger('stdout'), StreamLogger('stderr'):
|
||||
run("""until sudo fpga-describe-local-image -S {} -R -H | grep -q "loaded"; do sleep 1; done""".format(slotno))
|
||||
|
||||
def load_edma(self):
|
||||
""" load the edma kernel module. """
|
||||
self.instance_logger("Loading EDMA Driver Kernel Module.")
|
||||
for slotno in range(self.parentnode.get_num_fpga_slots_consumed(), self.parentnode.get_num_fpga_slots_max()):
|
||||
self.instance_logger("""Checking for Flashed FPGA Slot: {} with agfi: {}.""".format(slotno, dummyagfi))
|
||||
with StreamLogger('stdout'), StreamLogger('stderr'):
|
||||
run("""until sudo fpga-describe-local-image -S {} -R -H | grep -q "loaded"; do sleep 1; done""".format(slotno))
|
||||
|
||||
|
||||
def load_xdma(self):
|
||||
""" load the xdma kernel module. """
|
||||
# fpga mgmt tools seem to force load xocl after a flash now...
|
||||
# xocl conflicts with the xdma driver, which we actually want to use
|
||||
# so we just remove everything for good measure before loading xdma:
|
||||
self.unload_xdma()
|
||||
# now load xdma
|
||||
self.instance_logger("Loading XDMA Driver Kernel Module.")
|
||||
# TODO: can make these values automatically be chosen based on link lat
|
||||
with StreamLogger('stdout'), StreamLogger('stderr'):
|
||||
run("sudo insmod /home/centos/edma/linux_kernel_drivers/edma/edma-drv.ko single_transaction_size=65536 transient_buffer_size=67108864 edma_queue_depth=1024 poll_mode=1")
|
||||
run("sudo insmod /home/centos/xdma/linux_kernel_drivers/xdma/xdma.ko poll_mode=1")
|
||||
|
||||
def start_ila_server(self):
|
||||
""" start the vivado hw_server and virtual jtag on simulation instance.) """
|
||||
|
|
@ -396,9 +489,10 @@ class InstanceDeployManager:
|
|||
|
||||
files_to_copy = serv.get_required_files_local_paths()
|
||||
for filename in files_to_copy:
|
||||
# here, filename is a pair of (local path, remote path)
|
||||
with StreamLogger('stdout'), StreamLogger('stderr'):
|
||||
# -z --inplace
|
||||
rsync_cap = rsync_project(local_dir=filename, remote_dir=remote_sim_rsync_dir,
|
||||
rsync_cap = rsync_project(local_dir=filename[0], remote_dir=remote_sim_rsync_dir + '/' + filename[1],
|
||||
ssh_opts="-o StrictHostKeyChecking=no", extra_opts="-L", capture=True)
|
||||
rootLogger.debug(rsync_cap)
|
||||
rootLogger.debug(rsync_cap.stderr)
|
||||
|
|
@ -471,17 +565,19 @@ class InstanceDeployManager:
|
|||
self.copy_sim_slot_infrastructure(slotno)
|
||||
|
||||
self.get_and_install_aws_fpga_sdk()
|
||||
# unload any existing edma
|
||||
self.unload_edma()
|
||||
# copy edma driver
|
||||
self.fpga_node_edma()
|
||||
# unload any existing edma/xdma/xocl
|
||||
self.unload_xdma()
|
||||
# copy xdma driver
|
||||
self.fpga_node_xdma()
|
||||
# load xdma
|
||||
self.load_xdma()
|
||||
|
||||
# clear/flash fpgas
|
||||
self.clear_fpgas()
|
||||
self.flash_fpgas()
|
||||
|
||||
# re-load EDMA
|
||||
self.load_edma()
|
||||
# re-load XDMA
|
||||
self.load_xdma()
|
||||
|
||||
#restart (or start form scratch) ila server
|
||||
self.kill_ila_server()
|
||||
|
|
|
|||
|
|
@ -100,18 +100,77 @@ class RuntimeHWConfig:
|
|||
# the sed is in there to get rid of newlines in runtime confs
|
||||
driver = self.get_local_driver_binaryname()
|
||||
runtimeconf = self.get_local_runtimeconf_binaryname()
|
||||
basecommand = """screen -S fsim{slotid} -d -m bash -c "script -f -c 'stty intr ^] && sudo ./{driver} +permissive $(sed \':a;N;$!ba;s/\\n/ /g\' {runtimeconf}) +macaddr0={macaddr} +blkdev0={blkdev} +slotid={slotid} +niclog0=niclog {tracefile} +trace-start0={trace_start} +trace-end0={trace_end} +linklatency0={linklatency} +netbw0={netbw} +profile-interval=-1 +profile-interval={profile_interval} +zero-out-dram +shmemportname0={shmemportname} +permissive-off {bootbin} && stty intr ^c' uartlog"; sleep 1""".format(
|
||||
slotid=slotid, driver=driver, runtimeconf=runtimeconf,
|
||||
macaddr=macaddr, blkdev=blkdev, linklatency=linklatency,
|
||||
netbw=netbw, profile_interval=profile_interval,
|
||||
shmemportname=shmemportname, bootbin=bootbin, tracefile=tracefile,
|
||||
trace_start=trace_start, trace_end=trace_end)
|
||||
|
||||
driverArgs = """+permissive $(sed \':a;N;$!ba;s/\\n/ /g\' {runtimeconf}) +macaddr0={macaddr} +slotid={slotid} +niclog0=niclog {tracefile} +trace-start0={trace_start} +trace-end0={trace_end} +linklatency0={linklatency} +netbw0={netbw} +profile-interval={profile_interval} +zero-out-dram +shmemportname0={shmemportname} +permissive-off +prog0={bootbin}""".format(
|
||||
slotid=slotid, runtimeconf=runtimeconf, macaddr=macaddr,
|
||||
linklatency=linklatency, netbw=netbw,
|
||||
profile_interval=profile_interval, shmemportname=shmemportname,
|
||||
bootbin=bootbin, tracefile=tracefile, trace_start=trace_start,
|
||||
trace_end=trace_end)
|
||||
|
||||
if blkdev is not None:
|
||||
driverArgs += """ +blkdev0={blkdev}""".format(blkdev=blkdev)
|
||||
|
||||
basecommand = """screen -S fsim{slotid} -d -m bash -c "script -f -c 'stty intr ^] && sudo ./{driver} {driverArgs} && stty intr ^c' uartlog"; sleep 1""".format(
|
||||
slotid=slotid, driver=driver, driverArgs=driverArgs)
|
||||
|
||||
return basecommand
|
||||
|
||||
|
||||
def get_supernode_boot_simulation_command(self, slotid, all_macs,
|
||||
all_rootfses, all_linklatencies,
|
||||
all_netbws, profile_interval,
|
||||
all_bootbinaries, trace_enable,
|
||||
trace_start, trace_end,
|
||||
all_shmemportnames):
|
||||
""" return the command used to boot the simulation. this has to have
|
||||
some external params passed to it, because not everything is contained
|
||||
in a runtimehwconfig. TODO: maybe runtimehwconfig should be renamed to
|
||||
pre-built runtime config? It kinda contains a mix of pre-built and
|
||||
runtime parameters currently. """
|
||||
|
||||
tracefile = "+tracefile0=TRACEFILE" if trace_enable else ""
|
||||
|
||||
# this monstrosity boots the simulator, inside screen, inside script
|
||||
# the sed is in there to get rid of newlines in runtime confs
|
||||
driver = self.get_local_driver_binaryname()
|
||||
runtimeconf = self.get_local_runtimeconf_binaryname()
|
||||
|
||||
def array_to_plusargs(valuesarr, plusarg):
|
||||
args = []
|
||||
for index, arg in enumerate(valuesarr):
|
||||
if arg is not None:
|
||||
args.append("""{}{}={}""".format(plusarg, index, arg))
|
||||
return " ".join(args) + " "
|
||||
|
||||
command_macs = array_to_plusargs(all_macs, "+macaddr")
|
||||
command_rootfses = array_to_plusargs(all_rootfses, "+blkdev")
|
||||
command_linklatencies = array_to_plusargs(all_linklatencies, "+linklatency")
|
||||
command_netbws = array_to_plusargs(all_netbws, "+netbw")
|
||||
command_shmemportnames = array_to_plusargs(all_shmemportnames, "+shmemportname")
|
||||
|
||||
command_bootbinaries = array_to_plusargs(all_bootbinaries, "+prog")
|
||||
|
||||
|
||||
basecommand = """screen -S fsim{slotid} -d -m bash -c "script -f -c 'stty intr ^] && sudo ./{driver} +permissive $(sed \':a;N;$!ba;s/\\n/ /g\' {runtimeconf}) +slotid={slotid} +profile-interval={profile_interval} +zero-out-dram {command_macs} {command_rootfses} +niclog0=niclog {tracefile} +trace-start0={trace_start} +trace-end0={trace_end} {command_linklatencies} {command_netbws} {command_shmemportnames} +permissive-off {command_bootbinaries} && stty intr ^c' uartlog"; sleep 1""".format(
|
||||
slotid=slotid, driver=driver, runtimeconf=runtimeconf,
|
||||
command_macs=command_macs,
|
||||
command_rootfses=command_rootfses,
|
||||
command_linklatencies=command_linklatencies,
|
||||
command_netbws=command_netbws,
|
||||
profile_interval=profile_interval,
|
||||
command_shmemportnames=command_shmemportnames,
|
||||
command_bootbinaries=command_bootbinaries,
|
||||
trace_start=trace_start, trace_end=trace_end, tracefile=tracefile)
|
||||
|
||||
return basecommand
|
||||
|
||||
|
||||
|
||||
def get_kill_simulation_command(self):
|
||||
driver = self.get_local_driver_binaryname()
|
||||
return """sudo pkill -SIGKILL {driver}""".format(driver=driver)
|
||||
# Note that pkill only works for names <=15 characters
|
||||
return """sudo pkill -SIGKILL {driver}""".format(driver=driver[:15])
|
||||
|
||||
|
||||
def build_fpga_driver(self):
|
||||
|
|
@ -126,9 +185,16 @@ class RuntimeHWConfig:
|
|||
platform_config = triplet_pieces[2]
|
||||
rootLogger.info("Building FPGA software driver for " + str(self.get_deploytriplet_for_config()))
|
||||
with prefix('cd ../'), prefix('source sourceme-f1-manager.sh'), prefix('cd sim/'), StreamLogger('stdout'), StreamLogger('stderr'):
|
||||
localcap = local("""make DESIGN={} TARGET_CONFIG={} PLATFORM_CONFIG={} f1""".format(design, target_config, platform_config), capture=True)
|
||||
localcap = None
|
||||
with settings(warn_only=True):
|
||||
driverbuildcommand = """make DESIGN={} TARGET_CONFIG={} PLATFORM_CONFIG={} f1""".format(design, target_config, platform_config)
|
||||
localcap = local(driverbuildcommand, capture=True)
|
||||
rootLogger.debug("[localhost] " + str(localcap))
|
||||
rootLogger.debug("[localhost] " + str(localcap.stderr))
|
||||
if localcap.failed:
|
||||
rootLogger.info("FPGA software driver build failed. Exiting. See log for details.")
|
||||
rootLogger.info("""You can also re-run '{}' in the 'firesim/sim' directory to debug this error.""".format(driverbuildcommand))
|
||||
exit(1)
|
||||
|
||||
self.driver_built = True
|
||||
|
||||
|
|
@ -177,9 +243,10 @@ class InnerRuntimeConfiguration:
|
|||
runtime_dict[overridesection][overridefield] = overridevalue
|
||||
|
||||
self.runfarmtag = runtime_dict['runfarm']['runfarmtag']
|
||||
self.f1_16xlarges_requested = int(runtime_dict['runfarm']['f1_16xlarges'])
|
||||
self.m4_16xlarges_requested = int(runtime_dict['runfarm']['m4_16xlarges'])
|
||||
self.f1_2xlarges_requested = int(runtime_dict['runfarm']['f1_2xlarges'])
|
||||
self.f1_16xlarges_requested = int(runtime_dict['runfarm']['f1_16xlarges']) if 'f1_16xlarges' in runtime_dict['runfarm'] else 0
|
||||
self.f1_4xlarges_requested = int(runtime_dict['runfarm']['f1_4xlarges']) if 'f1_4xlarges' in runtime_dict['runfarm'] else 0
|
||||
self.m4_16xlarges_requested = int(runtime_dict['runfarm']['m4_16xlarges']) if 'm4_16xlarges' in runtime_dict['runfarm'] else 0
|
||||
self.f1_2xlarges_requested = int(runtime_dict['runfarm']['f1_2xlarges']) if 'f1_2xlarges' in runtime_dict['runfarm'] else 0
|
||||
|
||||
self.run_instance_market = runtime_dict['runfarm']['runinstancemarket']
|
||||
self.spot_interruption_behavior = runtime_dict['runfarm']['spotinterruptionbehavior']
|
||||
|
|
@ -234,6 +301,7 @@ class RuntimeConfig:
|
|||
self.workload = WorkloadConfig(self.innerconf.workload_name, self.launch_time)
|
||||
|
||||
self.runfarm = RunFarm(self.innerconf.f1_16xlarges_requested,
|
||||
self.innerconf.f1_4xlarges_requested,
|
||||
self.innerconf.f1_2xlarges_requested,
|
||||
self.innerconf.m4_16xlarges_requested,
|
||||
self.innerconf.runfarmtag,
|
||||
|
|
@ -255,10 +323,10 @@ class RuntimeConfig:
|
|||
""" directly called by top-level launchrunfarm command. """
|
||||
self.runfarm.launch_run_farm()
|
||||
|
||||
def terminate_run_farm(self, terminatesomef1_16, terminatesomef1_2,
|
||||
def terminate_run_farm(self, terminatesomef1_16, terminatesomef1_4, terminatesomef1_2,
|
||||
terminatesomem4_16, forceterminate):
|
||||
""" directly called by top-level terminaterunfarm command. """
|
||||
self.runfarm.terminate_run_farm(terminatesomef1_16, terminatesomef1_2,
|
||||
self.runfarm.terminate_run_farm(terminatesomef1_16, terminatesomef1_4, terminatesomef1_2,
|
||||
terminatesomem4_16, forceterminate)
|
||||
|
||||
def infrasetup(self):
|
||||
|
|
|
|||
|
|
@ -120,8 +120,6 @@ class UserTopologies(object):
|
|||
|
||||
self.custom_mapper = custom_mapper
|
||||
|
||||
|
||||
|
||||
def example_multilink(self):
|
||||
self.roots = [FireSimSwitchNode()]
|
||||
midswitch = FireSimSwitchNode()
|
||||
|
|
@ -155,7 +153,6 @@ class UserTopologies(object):
|
|||
midswitches[0].add_downlinks([servers[0]])
|
||||
midswitches[1].add_downlinks([servers[1]])
|
||||
|
||||
|
||||
def small_hierarchy_8sims(self):
|
||||
self.custom_mapper = 'mapping_use_one_f1_16xlarge'
|
||||
self.roots = [FireSimSwitchNode()]
|
||||
|
|
@ -165,7 +162,6 @@ class UserTopologies(object):
|
|||
for swno in range(len(midlevel)):
|
||||
midlevel[swno].add_downlinks(servers[swno])
|
||||
|
||||
|
||||
def small_hierarchy_2sims(self):
|
||||
self.custom_mapper = 'mapping_use_one_f1_16xlarge'
|
||||
self.roots = [FireSimSwitchNode()]
|
||||
|
|
@ -175,7 +171,6 @@ class UserTopologies(object):
|
|||
for swno in range(len(midlevel)):
|
||||
midlevel[swno].add_downlinks(servers[swno])
|
||||
|
||||
|
||||
def example_1config(self):
|
||||
self.roots = [FireSimSwitchNode()]
|
||||
servers = [FireSimServerNode() for y in range(1)]
|
||||
|
|
@ -259,6 +254,98 @@ class UserTopologies(object):
|
|||
for switchno in range(len(level2switches[switchgroupno])):
|
||||
level2switches[switchgroupno][switchno].add_downlinks(servers[switchgroupno][switchno])
|
||||
|
||||
@staticmethod
|
||||
def supernode_flatten(arr):
|
||||
res = []
|
||||
for x in arr:
|
||||
res = res + x
|
||||
return res
|
||||
|
||||
def supernode_example_6config(self):
|
||||
self.roots = [FireSimSwitchNode()]
|
||||
servers = [FireSimSuperNodeServerNode()] + [FireSimDummyServerNode() for x in range(5)]
|
||||
self.roots[0].add_downlinks(servers)
|
||||
|
||||
def supernode_example_4config(self):
|
||||
self.roots = [FireSimSwitchNode()]
|
||||
servers = [FireSimSuperNodeServerNode()] + [FireSimDummyServerNode() for x in range(3)]
|
||||
self.roots[0].add_downlinks(servers)
|
||||
def supernode_example_8config(self):
|
||||
self.roots = [FireSimSwitchNode()]
|
||||
servers = UserTopologies.supernode_flatten([[FireSimSuperNodeServerNode(), FireSimDummyServerNode(), FireSimDummyServerNode(), FireSimDummyServerNode()] for y in range(2)])
|
||||
self.roots[0].add_downlinks(servers)
|
||||
def supernode_example_16config(self):
|
||||
self.roots = [FireSimSwitchNode()]
|
||||
servers = UserTopologies.supernode_flatten([[FireSimSuperNodeServerNode(), FireSimDummyServerNode(), FireSimDummyServerNode(), FireSimDummyServerNode()] for y in range(4)])
|
||||
self.roots[0].add_downlinks(servers)
|
||||
def supernode_example_32config(self):
|
||||
self.roots = [FireSimSwitchNode()]
|
||||
servers = UserTopologies.supernode_flatten([[FireSimSuperNodeServerNode(), FireSimDummyServerNode(), FireSimDummyServerNode(), FireSimDummyServerNode()] for y in range(8)])
|
||||
self.roots[0].add_downlinks(servers)
|
||||
|
||||
def supernode_example_64config(self):
|
||||
self.roots = [FireSimSwitchNode()]
|
||||
level2switches = [FireSimSwitchNode() for x in range(2)]
|
||||
servers = [UserTopologies.supernode_flatten([[FireSimSuperNodeServerNode(), FireSimDummyServerNode(), FireSimDummyServerNode(), FireSimDummyServerNode()] for y in range(8)]) for x in range(2)]
|
||||
for root in self.roots:
|
||||
root.add_downlinks(level2switches)
|
||||
for l2switchNo in range(len(level2switches)):
|
||||
level2switches[l2switchNo].add_downlinks(servers[l2switchNo])
|
||||
|
||||
def supernode_example_128config(self):
|
||||
self.roots = [FireSimSwitchNode()]
|
||||
level2switches = [FireSimSwitchNode() for x in range(4)]
|
||||
servers = [UserTopologies.supernode_flatten([[FireSimSuperNodeServerNode(), FireSimDummyServerNode(), FireSimDummyServerNode(), FireSimDummyServerNode()] for y in range(8)]) for x in range(4)]
|
||||
for root in self.roots:
|
||||
root.add_downlinks(level2switches)
|
||||
for l2switchNo in range(len(level2switches)):
|
||||
level2switches[l2switchNo].add_downlinks(servers[l2switchNo])
|
||||
|
||||
def supernode_example_256config(self):
|
||||
self.roots = [FireSimSwitchNode()]
|
||||
level2switches = [FireSimSwitchNode() for x in range(8)]
|
||||
servers = [UserTopologies.supernode_flatten([[FireSimSuperNodeServerNode(), FireSimDummyServerNode(), FireSimDummyServerNode(), FireSimDummyServerNode()] for y in range(8)]) for x in range(8)]
|
||||
for root in self.roots:
|
||||
root.add_downlinks(level2switches)
|
||||
for l2switchNo in range(len(level2switches)):
|
||||
level2switches[l2switchNo].add_downlinks(servers[l2switchNo])
|
||||
|
||||
def supernode_example_512config(self):
|
||||
self.roots = [FireSimSwitchNode()]
|
||||
level1switches = [FireSimSwitchNode() for x in range(2)]
|
||||
level2switches = [[FireSimSwitchNode() for x in range(8)] for x in range(2)]
|
||||
servers = [[UserTopologies.supernode_flatten([[FireSimSuperNodeServerNode(), FireSimDummyServerNode(), FireSimDummyServerNode(), FireSimDummyServerNode()] for y in range(8)]) for x in range(8)] for x in range(2)]
|
||||
self.roots[0].add_downlinks(level1switches)
|
||||
for switchno in range(len(level1switches)):
|
||||
level1switches[switchno].add_downlinks(level2switches[switchno])
|
||||
for switchgroupno in range(len(level2switches)):
|
||||
for switchno in range(len(level2switches[switchgroupno])):
|
||||
level2switches[switchgroupno][switchno].add_downlinks(servers[switchgroupno][switchno])
|
||||
|
||||
def supernode_example_1024config(self):
|
||||
self.roots = [FireSimSwitchNode()]
|
||||
level1switches = [FireSimSwitchNode() for x in range(4)]
|
||||
level2switches = [[FireSimSwitchNode() for x in range(8)] for x in range(4)]
|
||||
servers = [[UserTopologies.supernode_flatten([[FireSimSuperNodeServerNode(), FireSimDummyServerNode(), FireSimDummyServerNode(), FireSimDummyServerNode()] for y in range(8)]) for x in range(8)] for x in range(4)]
|
||||
self.roots[0].add_downlinks(level1switches)
|
||||
for switchno in range(len(level1switches)):
|
||||
level1switches[switchno].add_downlinks(level2switches[switchno])
|
||||
for switchgroupno in range(len(level2switches)):
|
||||
for switchno in range(len(level2switches[switchgroupno])):
|
||||
level2switches[switchgroupno][switchno].add_downlinks(servers[switchgroupno][switchno])
|
||||
|
||||
def supernode_example_deep64config(self):
|
||||
self.roots = [FireSimSwitchNode()]
|
||||
level1switches = [FireSimSwitchNode() for x in range(2)]
|
||||
level2switches = [[FireSimSwitchNode() for x in range(1)] for x in range(2)]
|
||||
servers = [[UserTopologies.supernode_flatten([[FireSimSuperNodeServerNode(), FireSimDummyServerNode(), FireSimDummyServerNode(), FireSimDummyServerNode()] for y in range(8)]) for x in range(1)] for x in range(2)]
|
||||
self.roots[0].add_downlinks(level1switches)
|
||||
for switchno in range(len(level1switches)):
|
||||
level1switches[switchno].add_downlinks(level2switches[switchno])
|
||||
for switchgroupno in range(len(level2switches)):
|
||||
for switchno in range(len(level2switches[switchgroupno])):
|
||||
level2switches[switchgroupno][switchno].add_downlinks(servers[switchgroupno][switchno])
|
||||
|
||||
def dual_example_8config(self):
|
||||
""" two separate 8-node clusters for experiments, e.g. memcached mutilate. """
|
||||
self.roots = [FireSimSwitchNode(), FireSimSwitchNode()]
|
||||
|
|
@ -279,3 +366,4 @@ class UserTopologies(object):
|
|||
|
||||
def no_net_config(self):
|
||||
self.roots = [FireSimServerNode() for x in range(self.no_net_num_nodes)]
|
||||
|
||||
|
|
|
|||
|
|
@ -30,15 +30,30 @@ class JobConfig:
|
|||
else:
|
||||
self.bootbinary = parent_workload.common_bootbinary
|
||||
|
||||
if 'rootfs' in singlejob_dict:
|
||||
if singlejob_dict['rootfs'] is None:
|
||||
# Don't include a rootfs
|
||||
self.rootfs = None
|
||||
else:
|
||||
# Explicit per-job rootfs
|
||||
self.rootfs = parent_workload.workload_input_base_dir + singlejob_dict['rootfs']
|
||||
else:
|
||||
# No explicit per-job rootfs, inherit from workload
|
||||
if parent_workload.derive_rootfs:
|
||||
# No explicit workload rootfs, derive path from job name
|
||||
self.rootfs = self.parent_workload.workload_input_base_dir + self.jobname + self.filesystemsuffix
|
||||
elif parent_workload.common_rootfs is None:
|
||||
# Don't include a rootfs
|
||||
self.rootfs = None
|
||||
else:
|
||||
# Explicit rootfs path from workload
|
||||
self.rootfs = self.parent_workload.workload_input_base_dir + self.parent_workload.common_rootfs
|
||||
|
||||
def bootbinary_path(self):
|
||||
return self.parent_workload.workload_input_base_dir + self.bootbinary
|
||||
|
||||
def rootfs_path(self):
|
||||
if self.parent_workload.common_rootfs is not None:
|
||||
return self.parent_workload.workload_input_base_dir + self.parent_workload.common_rootfs
|
||||
else:
|
||||
# assume the rootfs is named after the job
|
||||
return self.parent_workload.workload_input_base_dir + self.jobname + self.filesystemsuffix
|
||||
return self.rootfs
|
||||
|
||||
def __str__(self):
|
||||
return self.jobname
|
||||
|
|
@ -60,7 +75,13 @@ class WorkloadConfig:
|
|||
with open(self.workloadfilename) as json_data:
|
||||
workloadjson = json.load(json_data)
|
||||
|
||||
self.common_rootfs = workloadjson.get("common_rootfs")
|
||||
if 'common_rootfs' in workloadjson:
|
||||
self.common_rootfs = workloadjson["common_rootfs"]
|
||||
self.derive_rootfs = False
|
||||
else:
|
||||
self.common_rootfs = None
|
||||
self.derive_rootfs = True
|
||||
|
||||
self.common_bootbinary = workloadjson.get("common_bootbinary")
|
||||
self.workload_name = workloadjson.get("benchmark_name")
|
||||
#self.rootfs_base = workloadjson.get("deliver_dir")
|
||||
|
|
|
|||
|
|
@ -7,6 +7,7 @@ s3bucketname=firesim-AWSUSERNAME
|
|||
buildinstancemarket=ondemand
|
||||
spotinterruptionbehavior=terminate
|
||||
spotmaxprice=ondemand
|
||||
postbuildhook=
|
||||
|
||||
[builds]
|
||||
# this section references builds defined in config_build_recipes.ini
|
||||
|
|
@ -21,6 +22,10 @@ firesim-quadcore-nic-ddr3-llc4mb
|
|||
fireboom-singlecore-no-nic-ddr3-llc4mb
|
||||
#fireboom-singlecore-nic-lbp
|
||||
fireboom-singlecore-nic-ddr3-llc4mb
|
||||
#firesim-supernode-singlecore-nic-ddr3-llc4mb
|
||||
#firesim-supernode-quadcore-nic-ddr3-llc4mb
|
||||
firesim-supernode-singlecore-nic-lbp
|
||||
|
||||
|
||||
[agfistoshare]
|
||||
firesim-singlecore-no-nic-lbp
|
||||
|
|
@ -33,6 +38,9 @@ firesim-quadcore-nic-ddr3-llc4mb
|
|||
fireboom-singlecore-no-nic-ddr3-llc4mb
|
||||
#fireboom-singlecore-nic-lbp
|
||||
fireboom-singlecore-nic-ddr3-llc4mb
|
||||
#firesim-supernode-singlecore-nic-ddr3-llc4mb
|
||||
#firesim-supernode-quadcore-nic-ddr3-llc4mb
|
||||
firesim-supernode-singlecore-nic-lbp
|
||||
|
||||
[sharewithaccounts]
|
||||
somebodysname=123456789012
|
||||
|
|
|
|||
|
|
@ -15,7 +15,7 @@
|
|||
[firesim-singlecore-no-nic-lbp]
|
||||
DESIGN=FireSimNoNIC
|
||||
TARGET_CONFIG=FireSimRocketChipSingleCoreConfig
|
||||
PLATFORM_CONFIG=FireSimConfig
|
||||
PLATFORM_CONFIG=FireSimConfig160MHz
|
||||
instancetype=c4.4xlarge
|
||||
deploytriplet=None
|
||||
|
||||
|
|
@ -36,14 +36,21 @@ deploytriplet=None
|
|||
[firesim-quadcore-nic-ddr3-llc4mb]
|
||||
DESIGN=FireSim
|
||||
TARGET_CONFIG=FireSimRocketChipQuadCoreConfig
|
||||
PLATFORM_CONFIG=FireSimDDR3FRFCFSLLC4MBConfig
|
||||
PLATFORM_CONFIG=FireSimDDR3FRFCFSLLC4MBConfig90MHz
|
||||
instancetype=c4.4xlarge
|
||||
deploytriplet=None
|
||||
|
||||
[firesim-quadcore-no-nic-ddr3-llc4mb]
|
||||
DESIGN=FireSimNoNIC
|
||||
TARGET_CONFIG=FireSimRocketChipQuadCoreConfig
|
||||
PLATFORM_CONFIG=FireSimDDR3FRFCFSLLC4MBConfig
|
||||
PLATFORM_CONFIG=FireSimDDR3FRFCFSLLC4MBConfig90MHz
|
||||
instancetype=c4.4xlarge
|
||||
deploytriplet=None
|
||||
|
||||
[firesim-quadcore-no-nic-ddr3-llc4mb-3div]
|
||||
DESIGN=FireSimNoNIC
|
||||
TARGET_CONFIG=FireSimRocketChipQuadCoreConfig
|
||||
PLATFORM_CONFIG=FireSimDDR3FRFCFSLLC4MB3ClockDivConfig
|
||||
instancetype=c4.4xlarge
|
||||
deploytriplet=None
|
||||
|
||||
|
|
@ -58,7 +65,7 @@ deploytriplet=None
|
|||
[fireboom-singlecore-no-nic-ddr3-llc4mb]
|
||||
DESIGN=FireBoomNoNIC
|
||||
TARGET_CONFIG=FireSimBoomConfig
|
||||
PLATFORM_CONFIG=FireSimDDR3FRFCFSLLC4MBConfig
|
||||
PLATFORM_CONFIG=FireSimDDR3FRFCFSLLC4MBConfig90MHz
|
||||
instancetype=c4.4xlarge
|
||||
deploytriplet=None
|
||||
|
||||
|
|
@ -72,6 +79,43 @@ deploytriplet=None
|
|||
[fireboom-singlecore-nic-ddr3-llc4mb]
|
||||
DESIGN=FireBoom
|
||||
TARGET_CONFIG=FireSimBoomConfig
|
||||
PLATFORM_CONFIG=FireSimDDR3FRFCFSLLC4MBConfig
|
||||
PLATFORM_CONFIG=FireSimDDR3FRFCFSLLC4MBConfig90MHz
|
||||
instancetype=c4.4xlarge
|
||||
deploytriplet=None
|
||||
|
||||
[firesim-supernode-singlecore-nic-lbp]
|
||||
DESIGN=FireSimSupernode
|
||||
TARGET_CONFIG=SupernodeFireSimRocketChipConfig
|
||||
PLATFORM_CONFIG=FireSimConfig90MHz
|
||||
instancetype=c4.4xlarge
|
||||
deploytriplet=None
|
||||
|
||||
[firesim-supernode-quadcore-nic-lbp]
|
||||
DESIGN=FireSimSupernode
|
||||
TARGET_CONFIG=SupernodeFireSimRocketChipQuadCoreConfig
|
||||
PLATFORM_CONFIG=FireSimConfig75MHz
|
||||
instancetype=c4.4xlarge
|
||||
deploytriplet=None
|
||||
|
||||
[firesim-supernode-singlecore-nic-ddr3-llc4mb]
|
||||
DESIGN=FireSimSupernode
|
||||
TARGET_CONFIG=SupernodeFireSimRocketChipConfig
|
||||
PLATFORM_CONFIG=FireSimDDR3FRFCFSLLC4MBConfig90MHz
|
||||
instancetype=c4.4xlarge
|
||||
deploytriplet=None
|
||||
|
||||
[firesim-supernode-quadcore-nic-ddr3-llc4mb]
|
||||
DESIGN=FireSimSupernode
|
||||
TARGET_CONFIG=SupernodeFireSimRocketChipQuadCoreConfig
|
||||
PLATFORM_CONFIG=FireSimDDR3FRFCFSLLC4MBConfig75MHz
|
||||
instancetype=c4.4xlarge
|
||||
deploytriplet=None
|
||||
|
||||
# MIDAS Examples -- BUILD SUPPORT ONLY; Can't launch driver correctly on runfarm
|
||||
[midasexamples-gcd]
|
||||
TARGET_PROJECT=midasexamples
|
||||
DESIGN=GCD
|
||||
TARGET_CONFIG=NoConfig
|
||||
PLATFORM_CONFIG=DefaultF1Config
|
||||
instancetype=c4.4xlarge
|
||||
deploytriplet=None
|
||||
|
|
|
|||
|
|
@ -10,27 +10,32 @@
|
|||
# own images.
|
||||
|
||||
[fireboom-singlecore-nic-ddr3-llc4mb]
|
||||
agfi=agfi-090491454199fb160
|
||||
agfi=agfi-02abda61b5764288c
|
||||
deploytripletoverride=None
|
||||
customruntimeconfig=None
|
||||
|
||||
[fireboom-singlecore-no-nic-ddr3-llc4mb]
|
||||
agfi=agfi-0df9101df7b7ff708
|
||||
agfi=agfi-084aa1fcc32be2dcb
|
||||
deploytripletoverride=None
|
||||
customruntimeconfig=None
|
||||
|
||||
[firesim-quadcore-nic-ddr3-llc4mb]
|
||||
agfi=agfi-030b49bce9bd5ef96
|
||||
agfi=agfi-0ca581654aeb2793d
|
||||
deploytripletoverride=None
|
||||
customruntimeconfig=None
|
||||
|
||||
[firesim-quadcore-no-nic-ddr3-llc4mb]
|
||||
agfi=agfi-06b9b705ab9af1238
|
||||
agfi=agfi-0fd2554e204e2b0e3
|
||||
deploytripletoverride=None
|
||||
customruntimeconfig=None
|
||||
|
||||
[firesim-singlecore-no-nic-lbp]
|
||||
agfi=agfi-0584a1a71df6a005a
|
||||
agfi=agfi-0594780e9a1888173
|
||||
deploytripletoverride=None
|
||||
customruntimeconfig=None
|
||||
|
||||
[firesim-supernode-singlecore-nic-lbp]
|
||||
agfi=agfi-064190ff13a98df35
|
||||
deploytripletoverride=None
|
||||
customruntimeconfig=None
|
||||
|
||||
|
|
|
|||
|
|
@ -6,6 +6,7 @@ runfarmtag=mainrunfarm
|
|||
|
||||
f1_16xlarges=1
|
||||
m4_16xlarges=0
|
||||
f1_4xlarges=0
|
||||
f1_2xlarges=0
|
||||
|
||||
runinstancemarket=ondemand
|
||||
|
|
|
|||
|
|
@ -1,6 +1,5 @@
|
|||
spec17-intrate
|
||||
spec17-intspeed
|
||||
gapbs
|
||||
build
|
||||
fedora-uniform/stage4-disk.img
|
||||
fedora-uniform/QEMU-ONLY-bbl
|
||||
|
|
@ -8,3 +7,5 @@ iperf3
|
|||
check-rtc-linux
|
||||
bw-test-one-instance/*.riscv
|
||||
bw-test-two-instances/*.riscv
|
||||
unittest/STRESSRUNS
|
||||
/gapbs
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
|
||||
# All the tests in here right now are based off the br-disk image.
|
||||
BASE_IMAGE:=../../sw/firesim-software/images/br-disk.img
|
||||
BASE_LINUX:=../../sw/firesim-software/images/br-disk-bin
|
||||
# All the tests in here right now are based off the br-baseimage.
|
||||
BASE_IMAGE:=../../sw/firesim-software/images/br-base.img
|
||||
BASE_LINUX:=../../sw/firesim-software/images/br-base-bin
|
||||
|
||||
# TODO: ideally we want to restructure this so that:
|
||||
# Proprietary benchmarks (e.g. spec) are available as separate disks that can
|
||||
|
|
@ -17,25 +17,42 @@ allpaper: memcached-thread-imbalance simperf-test-latency simperf-test-scale bw-
|
|||
|
||||
# Default to the submodule
|
||||
SPECKLE_DIR=Speckle
|
||||
GAP_DIR=runscripts/gapbs-scripts
|
||||
|
||||
#TODO: Provide runscripts for fp{speed, rate}
|
||||
spec17_suites = intrate intspeed
|
||||
spec17_rootfs_dirs := $(patsubst %, spec17-%, $(spec17-suites))
|
||||
|
||||
$(SPECKLE_DIR)/build/overlay/%:
|
||||
cd $(SPECKLE_DIR) && ./gen_binaries.sh --compile --suite $*
|
||||
#Default to ref input size for SPEC17
|
||||
spec17-%: input = ref
|
||||
|
||||
spec17-%: spec17-%.json $(SPECKLE_DIR)/build/overlay/%
|
||||
$(SPECKLE_DIR)/build/overlay/%/$(input):
|
||||
cd $(SPECKLE_DIR) && ./gen_binaries.sh --compile --suite $* --input $(input)
|
||||
|
||||
spec17-%: spec17-%.json $(SPECKLE_DIR)/build/overlay/%/$(input)
|
||||
mkdir -p $@
|
||||
cp $(BASE_LINUX) $@/bbl-vmlinux
|
||||
python gen-benchmark-rootfs.py -w $< -r -b $(BASE_IMAGE) \
|
||||
-s $(SPECKLE_DIR)/build/overlay/$*
|
||||
-s $(SPECKLE_DIR)/build/overlay/$*/$(input)
|
||||
|
||||
#Default to test input size for GAPBS
|
||||
gapbs: input = graph500
|
||||
|
||||
$(GAP_DIR)/overlay/$(input):
|
||||
cd $(GAP_DIR) && ./gen_run_scripts.sh --binaries --input $(input)
|
||||
|
||||
|
||||
gapbs: gapbs.json $(GAP_DIR)/overlay/$(input)
|
||||
mkdir -p $@
|
||||
cp $(BASE_LINUX) $@/bbl-vmlinux
|
||||
python gen-benchmark-rootfs.py -w $< -r -b $(BASE_IMAGE) \
|
||||
-s $(GAP_DIR)/overlay/$(input) \
|
||||
|
||||
memcached-thread-imbalance:
|
||||
mkdir -p $@
|
||||
sudo yum -y install gengetopt
|
||||
sudo pip install matplotlib
|
||||
sudo pip install pandas
|
||||
sudo pip2 install matplotlib
|
||||
sudo pip2 install pandas
|
||||
cd $@ && git submodule update --init mutilate-loadgen-riscv-release
|
||||
cd $@/mutilate-loadgen-riscv-release && ./build.sh
|
||||
python gen-benchmark-rootfs.py -w $@.json -r -b $(BASE_IMAGE) -s $@/mutilate-loadgen-riscv-release/overlay
|
||||
|
|
@ -60,6 +77,8 @@ simperf-test-scale: simperf-test
|
|||
|
||||
simperf-test-latency: simperf-test
|
||||
|
||||
flash-stress: simperf-test-latency
|
||||
|
||||
iperf3: iperf3.json
|
||||
mkdir -p $@
|
||||
cd $@ && ln -sf ../$(BASE_LINUX) bbl-vmlinux
|
||||
|
|
@ -78,7 +97,13 @@ check-rtc-linux:
|
|||
checksum-test:
|
||||
cd ../../target-design/firechip/tests && make checksum.riscv
|
||||
|
||||
ccbench-cache-sweep:
|
||||
cd ccbench-cache-sweep/ccbench/caches && make ARCH=riscv
|
||||
python gen-benchmark-rootfs.py -w $@.json -r -b $(BASE_IMAGE) -s $@/
|
||||
|
||||
|
||||
.PHONY: $(spec17_overlays) $(spec17_rootfs_dirs) gapbs fedora-uniform \
|
||||
memcached-thread-imbalance bw-test-one-instance bw-test-two-instances \
|
||||
ping-latency simperf-test simperf-test-latency simperf-test-scale \
|
||||
iperf3 check-rtc check-rtc-linux allpaper checksum-test
|
||||
iperf3 check-rtc check-rtc-linux allpaper checksum-test \
|
||||
ccbench-cache-sweep flash-stress
|
||||
|
|
|
|||
|
|
@ -1 +1 @@
|
|||
Subproject commit 75d0ba1e04e3acb159bf98ce0a9ed5866396599e
|
||||
Subproject commit f6503ea670599530011c964b45aa5bd31f6f1c31
|
||||
|
|
@ -0,0 +1,31 @@
|
|||
|
||||
[runfarm]
|
||||
runfarmtag=ccbench-cache-sweep
|
||||
|
||||
f1_16xlarges=0
|
||||
m4_16xlarges=0
|
||||
f1_2xlarges=1
|
||||
|
||||
runinstancemarket=ondemand
|
||||
spotinterruptionbehavior=terminate
|
||||
spotmaxprice=ondemand
|
||||
|
||||
[targetconfig]
|
||||
topology=no_net_config
|
||||
no_net_num_nodes=1
|
||||
# These are unused
|
||||
linklatency=6405
|
||||
switchinglatency=10
|
||||
netbandwidth=200
|
||||
profileinterval=-1
|
||||
|
||||
defaulthwconfig=firesim-quadcore-no-nic-ddr3-llc4mb
|
||||
|
||||
[tracing]
|
||||
enable=no
|
||||
startcycle=0
|
||||
endcycle=-1
|
||||
|
||||
[workload]
|
||||
workloadname=ccbench-cache-sweep.json
|
||||
terminateoncompletion=no
|
||||
|
|
@ -0,0 +1,19 @@
|
|||
{
|
||||
"common_bootbinary" : "bbl-vmlinux",
|
||||
"benchmark_name" : "ccbench-cache-sweep",
|
||||
"deliver_dir" : "/ccbench-cache-sweep/",
|
||||
"common_files" : ["runccbench.sh", "ccbench/caches/caches"],
|
||||
"common_args" : "",
|
||||
"common_outputs" : [],
|
||||
"common_simulation_outputs" : ["uartlog"],
|
||||
"post_run_hook": "python plotccbench.py",
|
||||
"workloads" : [
|
||||
{
|
||||
"name": "ccbench-all",
|
||||
"files": [],
|
||||
"command": "cd /ccbench-cache-sweep/ && ./runccbench.sh",
|
||||
"simulation_outputs": [],
|
||||
"outputs": []
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
@ -0,0 +1 @@
|
|||
ccbench-all.ext2
|
||||
|
|
@ -0,0 +1 @@
|
|||
../linux-uniform/br-base-bin
|
||||
|
|
@ -0,0 +1 @@
|
|||
Subproject commit a5b884a71287a7be509f8388342eb32b2de36a2a
|
||||
|
|
@ -0,0 +1,98 @@
|
|||
import matplotlib
|
||||
matplotlib.use('Agg')
|
||||
import matplotlib.pyplot as plt
|
||||
import pandas as pd
|
||||
import matplotlib.ticker as mticker
|
||||
import sys
|
||||
|
||||
starterpath = sys.argv[1]
|
||||
fname = starterpath + '/ccbench-all/uartlog'
|
||||
outputpath = starterpath + '/outputplot.pdf'
|
||||
|
||||
f = open(fname, 'r')
|
||||
q = f.readlines()
|
||||
f.close()
|
||||
|
||||
|
||||
q = filter(lambda x: x.startswith('App:'), q)
|
||||
q = map(lambda x: x.strip().split(","), q)
|
||||
q = map(lambda x: list(map(lambda z: z.split(":"), x)), q)
|
||||
|
||||
|
||||
def arr_to_dict(q):
|
||||
# to dicts
|
||||
as_dict = []
|
||||
for elem in q:
|
||||
d = dict()
|
||||
for pair in elem:
|
||||
d[pair[0]] = pair[1]
|
||||
as_dict.append(d)
|
||||
return as_dict
|
||||
|
||||
cacheline_stride_bmark = filter(lambda x: ['RunType', '[16]'] in x, q)
|
||||
unit_stride_bmark = filter(lambda x: ['RunType', '[1]'] in x, q)
|
||||
random_bmark = filter(lambda x: ['RunType', '[0]'] in x, q)
|
||||
|
||||
def data_from_full_dict(array_of_dict):
|
||||
times = []
|
||||
sizes = []
|
||||
for d in array_of_dict:
|
||||
time = eval(d['Time'])[0]
|
||||
appsize = eval(d['AppSize'])[0] * 4
|
||||
times.append(time)
|
||||
sizes.append(appsize)
|
||||
return {'size': sizes, 'time': times}
|
||||
|
||||
|
||||
|
||||
|
||||
cacheline_stride_bmark_data = data_from_full_dict(arr_to_dict(cacheline_stride_bmark))
|
||||
unit_stride_bmark_data = data_from_full_dict(arr_to_dict(unit_stride_bmark))
|
||||
random_bmark_data = data_from_full_dict(arr_to_dict(random_bmark))
|
||||
|
||||
cacheline_ccbench_df = pd.DataFrame(data=cacheline_stride_bmark_data)
|
||||
unit_ccbench_df = pd.DataFrame(data=unit_stride_bmark_data)
|
||||
random_ccbench_df = pd.DataFrame(data=random_bmark_data)
|
||||
|
||||
|
||||
cacheline_ccbench_df = cacheline_ccbench_df.sort_values(by=['size'])
|
||||
unit_ccbench_df = unit_ccbench_df.sort_values(by=['size'])
|
||||
random_ccbench_df = random_ccbench_df.sort_values(by=['size'])
|
||||
|
||||
|
||||
series = []
|
||||
cacheline_array_dim = list(cacheline_ccbench_df['size'])
|
||||
cacheline_array_time = list(cacheline_ccbench_df['time'])
|
||||
|
||||
unit_array_dim = list(unit_ccbench_df['size'])
|
||||
unit_array_time = list(unit_ccbench_df['time'])
|
||||
|
||||
random_array_dim = list(random_ccbench_df['size'])
|
||||
random_array_time = list(random_ccbench_df['time'])
|
||||
|
||||
|
||||
|
||||
fig, ax = plt.subplots()
|
||||
ser1, = plt.semilogx(random_array_dim, random_array_time, linestyle='--', marker='*', c='0.1', label='Random Stride')
|
||||
ser, = plt.semilogx(cacheline_array_dim, cacheline_array_time, linestyle='--', marker='^', c='0.7', label='Cacheline Stride')
|
||||
ser2, = plt.semilogx(unit_array_dim, unit_array_time, linestyle='--', marker='x', c='0.4', label='Unit Stride')
|
||||
|
||||
series.append(ser)
|
||||
series.append(ser1)
|
||||
series.append(ser2)
|
||||
|
||||
matplotlib.rcParams.update(matplotlib.rcParamsDefault)
|
||||
ax.set_xlabel(r'Array Dimension', size='12')
|
||||
ax.set_ylabel(r'Execution Time (cycles)', size='11')
|
||||
print(cacheline_stride_bmark_data['size'])
|
||||
ax.xaxis.set_major_formatter(mticker.ScalarFormatter())
|
||||
ax.xaxis.get_major_formatter().set_scientific(False)
|
||||
ax.xaxis.get_major_formatter().set_useOffset(False)
|
||||
plt.minorticks_off()
|
||||
ax.legend()
|
||||
ax.set_xticks(cacheline_stride_bmark_data['size'])
|
||||
ax.grid(linestyle='-', linewidth=0.3)
|
||||
plt.xticks(fontsize=8, rotation=90)
|
||||
fig = plt.gcf()
|
||||
fig.tight_layout()
|
||||
fig.savefig(outputpath, format='pdf')
|
||||
|
|
@ -0,0 +1,76 @@
|
|||
|
||||
mkdir /output
|
||||
chmod +x /ccbench-cache-sweep/ccbench/caches/caches
|
||||
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 16 3000 1
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 32 3000 1
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 64 3000 1
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 128 3000 1
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 256 3000 1
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 512 3000 1
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 1024 3000 1
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 2048 3000 1
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 4096 3000 1
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 8192 3000 1
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 16384 3000 1
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 32768 3000 1
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 65536 3000 1
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 131072 3000 1
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 262144 3000 1
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 524288 3000 1
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 1048576 3000 1
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 2097152 3000 1
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 4194304 3000 1
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 8388608 3000 1
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 16777216 3000 1
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 33554432 3000 1
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 67108864 3000 1
|
||||
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 16 3000 16
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 32 3000 16
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 64 3000 16
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 128 3000 16
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 256 3000 16
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 512 3000 16
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 1024 3000 16
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 2048 3000 16
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 4096 3000 16
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 8192 3000 16
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 16384 3000 16
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 32768 3000 16
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 65536 3000 16
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 131072 3000 16
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 262144 3000 16
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 524288 3000 16
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 1048576 3000 16
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 2097152 3000 16
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 4194304 3000 16
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 8388608 3000 16
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 16777216 3000 16
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 33554432 3000 16
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 67108864 3000 16
|
||||
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 16 3000 0
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 32 3000 0
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 64 3000 0
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 128 3000 0
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 256 3000 0
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 512 3000 0
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 1024 3000 0
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 2048 3000 0
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 4096 3000 0
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 8192 3000 0
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 16384 3000 0
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 32768 3000 0
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 65536 3000 0
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 131072 3000 0
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 262144 3000 0
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 524288 3000 0
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 1048576 3000 0
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 2097152 3000 0
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 4194304 3000 0
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 8388608 3000 0
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 16777216 3000 0
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 33554432 3000 0
|
||||
/ccbench-cache-sweep/ccbench/caches/caches 67108864 3000 0
|
||||
|
||||
|
|
@ -1,7 +1,7 @@
|
|||
{
|
||||
"benchmark_name" : "fedora-uniform",
|
||||
"common_bootbinary" : "fedora-disk-bin",
|
||||
"common_rootfs" : "fedora-disk.img",
|
||||
"common_bootbinary" : "fedora-base-bin",
|
||||
"common_rootfs" : "fedora-base.img",
|
||||
"common_outputs" : ["/etc/os-release"],
|
||||
"common_simulation_outputs" : ["uartlog"]
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1 @@
|
|||
../../../sw/firesim-software/images/fedora-base-bin
|
||||
|
|
@ -0,0 +1 @@
|
|||
../../../sw/firesim-software/images/fedora-base.img
|
||||
|
|
@ -1 +0,0 @@
|
|||
../../../sw/firesim-software/images/fedora-disk-bin
|
||||
|
|
@ -1 +0,0 @@
|
|||
../../../sw/firesim-software/images/fedora-disk.img
|
||||
|
|
@ -0,0 +1,31 @@
|
|||
[runfarm]
|
||||
runfarmtag=gapbs-runfarm
|
||||
|
||||
f1_16xlarges=0
|
||||
m4_16xlarges=0
|
||||
f1_2xlarges=6
|
||||
|
||||
runinstancemarket=ondemand
|
||||
spotinterruptionbehavior=terminate
|
||||
spotmaxprice=ondemand
|
||||
|
||||
[targetconfig]
|
||||
topology=no_net_config
|
||||
no_net_num_nodes=6
|
||||
# These are unused
|
||||
linklatency=6405
|
||||
switchinglatency=10
|
||||
netbandwidth=200
|
||||
profileinterval=-1
|
||||
|
||||
# Need not be single core.
|
||||
defaulthwconfig=firesim-quadcore-no-nic-ddr3-llc4mb
|
||||
|
||||
[tracing]
|
||||
enable=no
|
||||
startcyle=0
|
||||
endcycle=-1
|
||||
|
||||
[workload]
|
||||
workloadname=gapbs.json
|
||||
terminateoncompletion=yes
|
||||
|
|
@ -1,219 +1,46 @@
|
|||
{
|
||||
"common_bootbinary": "bbl-vmlinux",
|
||||
"common_bootbinary" : "bbl-vmlinux",
|
||||
"benchmark_name" : "gapbs",
|
||||
"deliver_dir" : "gabps",
|
||||
"common_args" : ["1"],
|
||||
"common_files" : ["gapbs.sh", "run"],
|
||||
"common_outputs" : ["/hpm_data"],
|
||||
"common_simulation_outputs" : ["uartlog"],
|
||||
"deliver_dir" : "gapbs",
|
||||
"common_args" : [],
|
||||
"common_files" : ["gapbs.sh"],
|
||||
"common_simulation_outputs": ["uartlog", "memory_stats.csv"],
|
||||
"common_outputs" : ["/output"],
|
||||
"workloads" : [
|
||||
{
|
||||
"name": "bfs-twitter",
|
||||
"files": ["bfs", "benchmark/graphs/twitter.sg"],
|
||||
"command": "cd /gapbs && ./gapbs.sh bfs-twitter",
|
||||
"simulation_outputs": [],
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"name": "pr-twitter",
|
||||
"files": ["pr", "benchmark/graphs/twitter.sg"],
|
||||
"command": "cd /gapbs && ./gapbs.sh pr-twitter",
|
||||
"simulation_outputs": [],
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"name": "cc-twitter",
|
||||
"files": ["cc", "benchmark/graphs/twitter.sg"],
|
||||
"command": "cd /gapbs && ./gapbs.sh cc-twitter",
|
||||
"simulation_outputs": [],
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"name": "bc-twitter",
|
||||
"files": ["bc", "benchmark/graphs/twitter.sg"],
|
||||
"command": "cd /gapbs && ./gapbs.sh bc-twitter",
|
||||
"simulation_outputs": [],
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"name": "bfs-web",
|
||||
"files": ["bfs", "benchmark/graphs/web.sg"],
|
||||
"command": "cd /gapbs && ./gapbs.sh bfs-web",
|
||||
"simulation_outputs": [],
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"name": "pr-web",
|
||||
"files": ["pr", "benchmark/graphs/web.sg"],
|
||||
"command": "cd /gapbs && ./gapbs.sh pr-web",
|
||||
"simulation_outputs": [],
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"name": "cc-web",
|
||||
"files": ["cc", "benchmark/graphs/web.sg"],
|
||||
"command": "cd /gapbs && ./gapbs.sh cc-web",
|
||||
"simulation_outputs": [],
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"name": "bc-web",
|
||||
"files": ["bc", "benchmark/graphs/web.sg"],
|
||||
"command": "cd /gapbs && ./gapbs.sh bc-web",
|
||||
"simulation_outputs": [],
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"name": "bfs-road",
|
||||
"files": ["bfs", "benchmark/graphs/road.sg"],
|
||||
"command": "cd /gapbs && ./gapbs.sh bfs-road",
|
||||
"simulation_outputs": [],
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"name": "pr-road",
|
||||
"files": ["pr", "benchmark/graphs/road.sg"],
|
||||
"command": "cd /gapbs && ./gapbs.sh pr-road",
|
||||
"simulation_outputs": [],
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"name": "cc-road",
|
||||
"files": ["cc", "benchmark/graphs/road.sg"],
|
||||
"command": "cd /gapbs && ./gapbs.sh cc-road",
|
||||
"simulation_outputs": [],
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"name": "bc-road",
|
||||
"files": ["bc", "benchmark/graphs/road.sg"],
|
||||
"command": "cd /gapbs && ./gapbs.sh bc-road",
|
||||
"simulation_outputs": [],
|
||||
"name": "bc-kron",
|
||||
"files": ["bc", "run/bc-kron.sh", "benchmark/graphs/kron.sg"],
|
||||
"command": "cd /gapbs && ./gapbs.sh bc-kron",
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"name": "bfs-kron",
|
||||
"files": ["bfs", "benchmark/graphs/kron.sg"],
|
||||
"files": ["bfs", "run/bfs-kron.sh", "benchmark/graphs/kron.sg"],
|
||||
"command": "cd /gapbs && ./gapbs.sh bfs-kron",
|
||||
"simulation_outputs": [],
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"name": "pr-kron",
|
||||
"files": ["pr", "benchmark/graphs/kron.sg"],
|
||||
"command": "cd /gapbs && ./gapbs.sh pr-kron",
|
||||
"simulation_outputs": [],
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"name": "cc-kron",
|
||||
"files": ["cc", "benchmark/graphs/kron.sg"],
|
||||
"files": ["cc", "run/cc-kron.sh", "benchmark/graphs/kron.sg"],
|
||||
"command": "cd /gapbs && ./gapbs.sh cc-kron",
|
||||
"simulation_outputs": [],
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"name": "bc-kron",
|
||||
"files": ["bc", "benchmark/graphs/kron.sg"],
|
||||
"command": "cd /gapbs && ./gapbs.sh bc-kron",
|
||||
"simulation_outputs": [],
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"name": "tc-kron",
|
||||
"files": ["tc", "benchmark/graphs/kronU.sg"],
|
||||
"command": "cd /gapbs && ./gapbs.sh tc-kron",
|
||||
"simulation_outputs": [],
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"name": "bfs-urand",
|
||||
"files": ["bfs", "benchmark/graphs/urand.sg"],
|
||||
"command": "cd /gapbs && ./gapbs.sh bfs-urand",
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"name": "pr-urand",
|
||||
"files": ["pr", "benchmark/graphs/urand.sg"],
|
||||
"command": "cd /gapbs && ./gapbs.sh pr-urand",
|
||||
"simulation_outputs": [],
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"name": "cc-urand",
|
||||
"files": ["cc", "benchmark/graphs/urand.sg"],
|
||||
"command": "cd /gapbs && ./gapbs.sh cc-urand",
|
||||
"simulation_outputs": [],
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"name": "bc-urand",
|
||||
"files": ["bc", "benchmark/graphs/urand.sg"],
|
||||
"command": "cd /gapbs && ./gapbs.sh bc-urand",
|
||||
"simulation_outputs": [],
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"name": "tc-urand",
|
||||
"files": ["tc", "benchmark/graphs/urandU.sg"],
|
||||
"command": "cd /gapbs && ./gapbs.sh tc-urand",
|
||||
"simulation_outputs": [],
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"name": "sssp-twitter",
|
||||
"files": ["sssp", "benchmark/graphs/twitter.wsg"],
|
||||
"command": "cd /gapbs && ./gapbs.sh sssp-twitter",
|
||||
"simulation_outputs": [],
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"name": "sssp-web",
|
||||
"files": ["sssp", "benchmark/graphs/web.wsg"],
|
||||
"command": "cd /gapbs && ./gapbs.sh sssp-web",
|
||||
"simulation_outputs": [],
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"name": "sssp-road",
|
||||
"files": ["sssp", "benchmark/graphs/road.wsg"],
|
||||
"command": "cd /gapbs && ./gapbs.sh sssp-road",
|
||||
"simulation_outputs": [],
|
||||
"name": "pr-kron",
|
||||
"files": ["pr", "run/pr-kron.sh", "benchmark/graphs/kron.sg"],
|
||||
"command": "cd /gapbs && ./gapbs.sh pr-kron",
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"name": "sssp-kron",
|
||||
"files": ["sssp", "benchmark/graphs/kron.wsg"],
|
||||
"files": ["sssp", "run/sssp-kron.sh", "benchmark/graphs/kron.wsg"],
|
||||
"command": "cd /gapbs && ./gapbs.sh sssp-kron",
|
||||
"simulation_outputs": [],
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"name": "sssp-urand",
|
||||
"files": ["sssp", "benchmark/graphs/urand.wsg"],
|
||||
"command": "cd /gapbs && ./gapbs.sh sssp-urand",
|
||||
"simulation_outputs": [],
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"name": "tc-twitter",
|
||||
"files": ["tc", "benchmark/graphs/twitterU.sg"],
|
||||
"command": "cd /gapbs && ./gapbs.sh tc-twitter",
|
||||
"simulation_outputs": [],
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"name": "tc-web",
|
||||
"files": ["tc", "benchmark/graphs/webU.sg"],
|
||||
"command": "cd /gapbs && ./gapbs.sh tc-web",
|
||||
"simulation_outputs": [],
|
||||
"outputs": []
|
||||
},
|
||||
{
|
||||
"name": "tc-road",
|
||||
"files": ["tc", "benchmark/graphs/roadU.sg"],
|
||||
"command": "cd /gapbs && ./gapbs.sh tc-road",
|
||||
"simulation_outputs": [],
|
||||
"name": "tc-kron",
|
||||
"files": ["tc", "run/tc-kron.sh", "benchmark/graphs/kronU.sg"],
|
||||
"command": "cd /gapbs && ./gapbs.sh tc-kron",
|
||||
"outputs": []
|
||||
}
|
||||
]
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
{
|
||||
"benchmark_name" : "linux-uniform",
|
||||
"common_bootbinary" : "br-disk-bin",
|
||||
"common_rootfs" : "br-disk.img",
|
||||
"common_bootbinary" : "br-base-bin",
|
||||
"common_rootfs" : "br-base.img",
|
||||
"common_outputs" : ["/etc/os-release"],
|
||||
"common_simulation_outputs" : ["uartlog", "memory_stats.csv"]
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1 @@
|
|||
../../../sw/firesim-software/images/br-base-bin
|
||||
|
|
@ -0,0 +1 @@
|
|||
../../../sw/firesim-software/images/br-base.img
|
||||
|
|
@ -1 +0,0 @@
|
|||
../../../sw/firesim-software/images/br-disk-bin
|
||||
|
|
@ -1 +0,0 @@
|
|||
../../../sw/firesim-software/images/br-disk.img
|
||||
|
|
@ -1 +1 @@
|
|||
../../../sw/firesim-software/images/br-disk-bin
|
||||
../linux-uniform/br-base-bin
|
||||
|
|
@ -1 +1 @@
|
|||
Subproject commit 328818b8813ee84f977c8a1a94456d735ad72432
|
||||
Subproject commit 150a77698e8e786b8a87c14ae383889a1c24df67
|
||||
|
|
@ -1 +1 @@
|
|||
../../../sw/firesim-software/images/br-disk-bin
|
||||
../linux-uniform/br-base-bin
|
||||
|
|
@ -0,0 +1,16 @@
|
|||
#!/usr/bin/env bash
|
||||
|
||||
# the runfarm WILL be terminated upon completion
|
||||
|
||||
trap "exit" INT
|
||||
set -e
|
||||
set -o pipefail
|
||||
|
||||
if [ "$1" == "withlaunch" ]; then
|
||||
firesim -c workloads/ccbench-cache-sweep.ini launchrunfarm
|
||||
fi
|
||||
|
||||
firesim -c workloads/ccbench-cache-sweep.ini infrasetup
|
||||
firesim -c workloads/ccbench-cache-sweep.ini runworkload
|
||||
firesim -c workloads/ccbench-cache-sweep.ini terminaterunfarm --forceterminate
|
||||
|
||||
|
|
@ -0,0 +1,75 @@
|
|||
#!/usr/bin/env bash
|
||||
|
||||
# IMPORTANT! availability zone placement MATTERS for simulation performance.
|
||||
# Usually, you will not be able to get 32 nodes in one availability zone,
|
||||
# but frequently you will be able to get 16. The way the manager currently
|
||||
# terminates nodes is based on their sorted ip address, which may means you
|
||||
# end up killing some from different avail zones, even though it would be
|
||||
# ideal to completely remove nodes from "extra" availability zones.
|
||||
#
|
||||
# Until this is fixed in the manager, you should run this script separately to:
|
||||
# 1) get the 256 node result by commenting out the rest of the calls to loopfunc
|
||||
# 2) terminate your run farm, then launch a new one (where all 16 hosts +
|
||||
# switches will likely be in the same availability zone) that supports only 128
|
||||
# nodes and below, then run from there, decreasing scale.
|
||||
|
||||
# run the simperf SCALE poweroff test using the manager. optionally passing "withlaunch" will also
|
||||
# automatically launch the appropriate runfarm
|
||||
#
|
||||
# the runfarm WILL NOT be terminated upon completion
|
||||
|
||||
trap "exit" INT
|
||||
set -e
|
||||
set -o pipefail
|
||||
|
||||
if [ "$1" == "withlaunch" ]; then
|
||||
firesim launchrunfarm -c workloads/simperf-test-scale-supernode-config.ini
|
||||
fi
|
||||
|
||||
ORIGDIR=$(pwd)
|
||||
|
||||
cd ../results-workload
|
||||
|
||||
# create the aggregate results directory
|
||||
resultsdir=$(date +"%Y-%m-%d--%H-%M-%S")-simperf-test-scale-supernode-aggregate
|
||||
mkdir $resultsdir
|
||||
|
||||
# make sure we don't get the same name as one of the manager produced results
|
||||
# directories
|
||||
sleep 2
|
||||
|
||||
loopfunc () {
|
||||
echo "RUNNING supernode_example_$1config"
|
||||
# arg 1 is num nodes
|
||||
# arg 2 is num f116xlarges to kill AFTERWARDS
|
||||
# arg 3 is num m416xlarges to kill AFTERWARDS
|
||||
firesim infrasetup -c workloads/simperf-test-scale-supernode-config.ini --overrideconfigdata "targetconfig topology supernode_example_$1config"
|
||||
firesim runworkload -c workloads/simperf-test-scale-supernode-config.ini --overrideconfigdata "targetconfig topology supernode_example_$1config"
|
||||
# rename the output directory with the ping latency
|
||||
files=(*simperf-test-scale*)
|
||||
originalfilename=${files[-1]}
|
||||
mv $originalfilename $resultsdir/$1
|
||||
|
||||
firesim terminaterunfarm -c workloads/simperf-test-scale-supernode-config.ini --terminatesomef116 $2 --terminatesomem416 $3 --forceterminate
|
||||
|
||||
}
|
||||
|
||||
loopfunc 1024 16 2
|
||||
|
||||
loopfunc 512 8 2
|
||||
|
||||
loopfunc 256 4 0
|
||||
|
||||
loopfunc 128 2 0
|
||||
|
||||
loopfunc 64 1 1
|
||||
|
||||
loopfunc 32 0 0
|
||||
|
||||
loopfunc 16 0 0
|
||||
|
||||
loopfunc 8 0 0
|
||||
|
||||
loopfunc 4 1 0
|
||||
|
||||
python $ORIGDIR/simperf-test-scale/simperf-test-scale-results.py $(pwd)/$resultsdir
|
||||
|
|
@ -0,0 +1,67 @@
|
|||
#!/usr/bin/env bash
|
||||
|
||||
# This is some sugar around:
|
||||
# ./firesim -c <ini> {launchrunfarm && infrasetup && runworkload && terminaterunfarm}
|
||||
# And thus will only work for workloads that do not need run other applications
|
||||
# between firesim calls
|
||||
|
||||
# Defaults
|
||||
withlaunch=0
|
||||
terminate=1
|
||||
|
||||
function usage
|
||||
{
|
||||
echo "usage: run-workload.sh <workload.ini> [-H | -h | --help] [--noterminate] [--withlaunch]"
|
||||
echo " workload.ini: the firesim-relative path to the workload you'd like to run"
|
||||
echo " e.g. workloads/gapbs.ini"
|
||||
echo " --withlaunch: (Optional) will spin up a runfarm based on the ini"
|
||||
echo " --noterminate: (Optional) will not forcibly terminate runfarm instances after runworkload"
|
||||
}
|
||||
|
||||
if [ $# -eq 0 -o "$1" == "--help" -o "$1" == "-h" -o "$1" == "-H" ]; then
|
||||
usage
|
||||
exit 3
|
||||
fi
|
||||
|
||||
ini=$1
|
||||
shift
|
||||
|
||||
while test $# -gt 0
|
||||
do
|
||||
case "$1" in
|
||||
--withlaunch)
|
||||
withlaunch=1
|
||||
;;
|
||||
--noterminate)
|
||||
terminate=0;
|
||||
;;
|
||||
-h | -H | -help)
|
||||
usage
|
||||
exit
|
||||
;;
|
||||
--*) echo "ERROR: bad option $1"
|
||||
usage
|
||||
exit 1
|
||||
;;
|
||||
*) echo "ERROR: bad argument $1"
|
||||
usage
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
shift
|
||||
done
|
||||
|
||||
trap "exit" INT
|
||||
set -e
|
||||
set -o pipefail
|
||||
|
||||
if [ "$withlaunch" -ne "0" ]; then
|
||||
firesim -c $ini launchrunfarm
|
||||
fi
|
||||
|
||||
firesim -c $ini infrasetup
|
||||
firesim -c $ini runworkload
|
||||
|
||||
if [ "$terminate" -eq "1" ]; then
|
||||
firesim -c $ini terminaterunfarm --forceterminate
|
||||
fi
|
||||
|
|
@ -0,0 +1 @@
|
|||
overlay/*
|
||||
|
|
@ -0,0 +1 @@
|
|||
Subproject commit 116c480d06cf1bd239c276f14ea3e24a466e06e1
|
||||
|
|
@ -0,0 +1,25 @@
|
|||
#!/bin/bash
|
||||
verify=0
|
||||
function usage
|
||||
{
|
||||
echo "usage: gapbs.sh <workload-name> [-H | -h | --help] [--verify]"
|
||||
echo " workload-name: the kernel and graph input"
|
||||
echo " verify: if set, verifies the output of the benchmark. Default is off"
|
||||
}
|
||||
|
||||
if [ $# -eq 0 -o "$1" == "--help" -o "$1" == "-h" ]; then
|
||||
usage
|
||||
exit 3
|
||||
fi
|
||||
|
||||
bmark_name=$1
|
||||
shift
|
||||
mkdir -p ~/output
|
||||
export OMP_NUM_THREADS=`grep -o 'hart' /proc/cpuinfo | wc -l`
|
||||
echo "Starting rate $bmark_name run with $OMP_NUM_THREADS threads"
|
||||
if [ "$1" == "--verify" ]; then
|
||||
echo "and verifying output."
|
||||
./run/${bmark_name}.sh -v > ~/output/out 2>~/output/err
|
||||
else
|
||||
./run/${bmark_name}.sh > ~/output/out 2>~/output/err
|
||||
fi
|
||||
|
|
@ -0,0 +1,109 @@
|
|||
#!/bin/bash
|
||||
#default values
|
||||
command_file=test.cmd
|
||||
t_pwd="/gapbs"
|
||||
jsonFlag=false
|
||||
input_type=test
|
||||
binariesFlag=false
|
||||
KRON_ARGS=-g10
|
||||
SUITE="bc bfs cc cc_sv pr sssp tc ../gapbs.sh"
|
||||
workload_file="gapbs.json"
|
||||
workload=$(basename $workload_file .json)
|
||||
|
||||
#default vals for json
|
||||
bootbinary="bbl-vmlinux"
|
||||
output="/benchmark/out/"
|
||||
root_fs="gapbs.img"
|
||||
|
||||
function usage
|
||||
{
|
||||
echo "usage: gen_run_scripts.sh [--binaries] [--json] [-h] --input [test | graph500 | ref]"
|
||||
}
|
||||
|
||||
while test $# -gt 0
|
||||
do
|
||||
case "$1" in
|
||||
--json)
|
||||
jsonFlag=true
|
||||
;;
|
||||
--input)
|
||||
shift;
|
||||
input_type=$1
|
||||
command_file=$1.cmd
|
||||
;;
|
||||
-h)
|
||||
usage
|
||||
exit
|
||||
;;
|
||||
--binaries)
|
||||
binariesFlag=true
|
||||
;;
|
||||
esac
|
||||
shift
|
||||
done
|
||||
if [ "$input_type" = graph500 ];
|
||||
then
|
||||
KRON_ARGS=-g20
|
||||
elif [ "$input_type" = ref ];
|
||||
then
|
||||
echo "ref currently not supported"
|
||||
exit
|
||||
KRON_ARGS=-g27
|
||||
fi
|
||||
|
||||
if [ "$binariesFlag" = true ] && [ ! -d overlay/$input_type ];
|
||||
then
|
||||
overlay_dir="../overlay/$input_type"
|
||||
cd gapbs
|
||||
make converter
|
||||
CXX=${RISCV}/bin/riscv64-unknown-linux-gnu-g++ CXX_FLAGS+=--static make
|
||||
mkdir -p $overlay_dir/benchmark/graphs
|
||||
cp $SUITE $overlay_dir/
|
||||
./converter $KRON_ARGS -wb $overlay_dir/benchmark/graphs/kron.wsg
|
||||
./converter $KRON_ARGS -b $overlay_dir/benchmark/graphs/kron.sg
|
||||
./converter $KRON_ARGS -b $overlay_dir/benchmark/graphs/kronU.sg
|
||||
cd ..
|
||||
fi
|
||||
overlay_dir="overlay/$input_type"
|
||||
|
||||
mkdir -p $overlay_dir/run
|
||||
if [ "$jsonFlag" = true ];
|
||||
then
|
||||
echo "{" > $workload_file
|
||||
echo " \"common_bootbinary\" : \"${bootbinary}\"," >> $workload_file
|
||||
echo " \"benchmark_name\" : \"gapbs\"," >> $workload_file
|
||||
echo " \"deliver_dir\" : \"${workload}\"," >> $workload_file
|
||||
echo " \"common_args\" : []," >> $workload_file
|
||||
echo " \"common_files\" : [\"gapbs.sh\"]," >> $workload_file
|
||||
echo " \"common_simulation_outputs:\" : [\"uartlog\"]," >> $workload_file
|
||||
echo " \"workloads\" : [" >> $workload_file
|
||||
fi
|
||||
|
||||
while IFS= read -r command; do
|
||||
bmark=`echo $command | sed 's/\.\/\([a-z]*\).*/\1/'`
|
||||
graph=`echo ${command} | grep -Eo 'benchmark/graphs/\w*\.\w*'`
|
||||
output_file="`echo $command | grep -Eo "benchmark\/out/.*out"`"
|
||||
workload=$(basename $output_file .out)
|
||||
binary="${bmark}"
|
||||
run_script=$overlay_dir/run/${workload}.sh
|
||||
run_script_no_overlay=run/${workload}.sh
|
||||
echo '#!/bin/bash' > $run_script
|
||||
#verify option always inserted into run_script
|
||||
echo $command | sed "s/benchmark/\\${t_pwd}\/benchmark/g" | sed "s/^\./\\${t_pwd}/" | sed "s/-n/\$1 -n/g" |sed "s/ >.*//" >> $run_script
|
||||
|
||||
chmod +x $run_script
|
||||
if [ "$jsonFlag" = true ]; then
|
||||
echo " {" >> $workload_file
|
||||
echo " \"name\": \"${workload}\"," >> $workload_file
|
||||
echo " \"files\": [\"${binary}\", \"${run_script_no_overlay}\", \"${graph}\"]," >> $workload_file
|
||||
echo " \"command\": \"cd /gapbs && ./gapbs.sh ${workload}\"," >> $workload_file
|
||||
echo " \"outputs\": []" >> $workload_file
|
||||
echo " }," >> $workload_file
|
||||
fi
|
||||
done < $command_file
|
||||
if [ "$jsonFlag" = true ]; then
|
||||
echo "$(head -n -1 $workload_file)" > $workload_file
|
||||
echo " }" >> $workload_file
|
||||
echo " ]" >> $workload_file
|
||||
echo "}" >> $workload_file
|
||||
fi
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
./bfs -f benchmark/graphs/kron.sg -n64 > benchmark/out/bfs-kron.out
|
||||
./pr -f benchmark/graphs/kron.sg -i1000 -t1e-4 -n16 > benchmark/out/pr-kron.out
|
||||
./cc -f benchmark/graphs/kron.sg -n16 > benchmark/out/cc-kron.out
|
||||
./bc -f benchmark/graphs/kron.sg -i4 -n16 > benchmark/out/bc-kron.out
|
||||
./tc -f benchmark/graphs/kronU.sg -n3 > benchmark/out/tc-kron.out
|
||||
./sssp -f benchmark/graphs/kron.wsg -n64 -d2 > benchmark/out/sssp-kron.out
|
||||
|
|
@ -0,0 +1,30 @@
|
|||
./bfs -f benchmark/graphs/twitter.sg -n64 > benchmark/out/bfs-twitter.out
|
||||
./pr -f benchmark/graphs/twitter.sg -i1000 -t1e-4 -n16 > benchmark/out/pr-twitter.out
|
||||
./cc -f benchmark/graphs/twitter.sg -n16 > benchmark/out/cc-twitter.out
|
||||
./bc -f benchmark/graphs/twitter.sg -i4 -n16 > benchmark/out/bc-twitter.out
|
||||
./bfs -f benchmark/graphs/web.sg -n64 > benchmark/out/bfs-web.out
|
||||
./pr -f benchmark/graphs/web.sg -i1000 -t1e-4 -n16 > benchmark/out/pr-web.out
|
||||
./cc -f benchmark/graphs/web.sg -n16 > benchmark/out/cc-web.out
|
||||
./bc -f benchmark/graphs/web.sg -i4 -n16 > benchmark/out/bc-web.out
|
||||
./bfs -f benchmark/graphs/road.sg -n64 > benchmark/out/bfs-road.out
|
||||
./pr -f benchmark/graphs/road.sg -i1000 -t1e-4 -n16 > benchmark/out/pr-road.out
|
||||
./cc -f benchmark/graphs/road.sg -n16 > benchmark/out/cc-road.out
|
||||
./bc -f benchmark/graphs/road.sg -i4 -n16 > benchmark/out/bc-road.out
|
||||
./bfs -f benchmark/graphs/kron.sg -n64 > benchmark/out/bfs-kron.out
|
||||
./pr -f benchmark/graphs/kron.sg -i1000 -t1e-4 -n16 > benchmark/out/pr-kron.out
|
||||
./cc -f benchmark/graphs/kron.sg -n16 > benchmark/out/cc-kron.out
|
||||
./bc -f benchmark/graphs/kron.sg -i4 -n16 > benchmark/out/bc-kron.out
|
||||
./tc -f benchmark/graphs/kronU.sg -n3 > benchmark/out/tc-kron.out
|
||||
./bfs -f benchmark/graphs/urand.sg -n64 > benchmark/out/bfs-urand.out
|
||||
./pr -f benchmark/graphs/urand.sg -i1000 -t1e-4 -n16 > benchmark/out/pr-urand.out
|
||||
./cc -f benchmark/graphs/urand.sg -n16 > benchmark/out/cc-urand.out
|
||||
./bc -f benchmark/graphs/urand.sg -i4 -n16 > benchmark/out/bc-urand.out
|
||||
./tc -f benchmark/graphs/urandU.sg -n3 > benchmark/out/tc-urand.out
|
||||
./sssp -f benchmark/graphs/twitter.wsg -n64 -d2 > benchmark/out/sssp-twitter.out
|
||||
./sssp -f benchmark/graphs/web.wsg -n64 -d2 > benchmark/out/sssp-web.out
|
||||
./sssp -f benchmark/graphs/road.wsg -n64 -d50000 > benchmark/out/sssp-road.out
|
||||
./sssp -f benchmark/graphs/kron.wsg -n64 -d2 > benchmark/out/sssp-kron.out
|
||||
./sssp -f benchmark/graphs/urand.wsg -n64 -d2 > benchmark/out/sssp-urand.out
|
||||
./tc -f benchmark/graphs/twitterU.sg -n3 > benchmark/out/tc-twitter.out
|
||||
./tc -f benchmark/graphs/webU.sg -n3 > benchmark/out/tc-web.out
|
||||
./tc -f benchmark/graphs/roadU.sg -n3 > benchmark/out/tc-road.out
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
./bc -f benchmark/graphs/kron.sg -n1 > benchmark/out/bc-kron.out
|
||||
./bfs -f benchmark/graphs/kron.sg -n1 > benchmark/out/bfs-kron.out
|
||||
./cc -f benchmark/graphs/kron.sg -n1 > benchmark/out/cc-kron.out
|
||||
./pr -f benchmark/graphs/kron.sg -n1 > benchmark/out/pr-kron.out
|
||||
./sssp -f benchmark/graphs/kron.wsg -n1 > benchmark/out/sssp-kron.out
|
||||
./tc -f benchmark/graphs/kronU.sg -n1 > benchmark/out/tc-kron.out
|
||||
|
|
@ -0,0 +1,24 @@
|
|||
[runfarm]
|
||||
runfarmtag=simperftestscalesupernode-mainrunfarm
|
||||
|
||||
f1_16xlarges=32
|
||||
m4_16xlarges=5
|
||||
f1_2xlarges=0
|
||||
|
||||
runinstancemarket=ondemand
|
||||
spotinterruptionbehavior=terminate
|
||||
spotmaxprice=ondemand
|
||||
|
||||
[targetconfig]
|
||||
topology=supernode_example_1024config
|
||||
no_net_num_nodes=2
|
||||
linklatency=6405
|
||||
switchinglatency=10
|
||||
netbandwidth=200
|
||||
profileinterval=-1
|
||||
|
||||
defaulthwconfig=firesim-supernode-singlecore-nic-lbp
|
||||
|
||||
[workload]
|
||||
workloadname=simperf-test-scale.json
|
||||
terminateoncompletion=no
|
||||
|
|
@ -1 +1 @@
|
|||
../../../sw/firesim-software/images/br-disk-bin
|
||||
../linux-uniform/br-base-bin
|
||||
|
|
@ -16,9 +16,15 @@ no_net_num_nodes=10
|
|||
linklatency=6405
|
||||
switchinglatency=10
|
||||
netbandwidth=200
|
||||
profileinterval=-1
|
||||
|
||||
defaulthwconfig=firesim-quadcore-no-nic-ddr3-llc4mb
|
||||
|
||||
[tracing]
|
||||
enable=no
|
||||
startcycle=0
|
||||
endcycle=-1
|
||||
|
||||
[workload]
|
||||
workloadname=spec17-intrate.json
|
||||
terminateoncompletion=yes
|
||||
|
|
|
|||
|
|
@ -16,10 +16,16 @@ no_net_num_nodes=11
|
|||
linklatency=6405
|
||||
switchinglatency=10
|
||||
netbandwidth=200
|
||||
profileinterval=-1
|
||||
|
||||
# Need not be single core.
|
||||
defaulthwconfig= firesim-singlecore-no-nic-lbp
|
||||
|
||||
[tracing]
|
||||
enable=no
|
||||
startcyle=0
|
||||
endcycle=-1
|
||||
|
||||
[workload]
|
||||
workloadname=spec17-intspeed.json
|
||||
terminateoncompletion=yes
|
||||
|
|
|
|||
|
|
@ -0,0 +1,24 @@
|
|||
[runfarm]
|
||||
runfarmtag=flash-stress
|
||||
|
||||
f1_16xlarges=1
|
||||
m4_16xlarges=0
|
||||
f1_2xlarges=0
|
||||
|
||||
runinstancemarket=ondemand
|
||||
spotinterruptionbehavior=terminate
|
||||
spotmaxprice=ondemand
|
||||
|
||||
[targetconfig]
|
||||
topology=example_8config
|
||||
no_net_num_nodes=2
|
||||
linklatency=6405
|
||||
switchinglatency=10
|
||||
netbandwidth=200
|
||||
profileinterval=-1
|
||||
|
||||
defaulthwconfig=firesim-quadcore-nic-ddr3-llc4mb
|
||||
|
||||
[workload]
|
||||
workloadname=unittest/flash-stress.json
|
||||
terminateoncompletion=no
|
||||
|
|
@ -0,0 +1,7 @@
|
|||
{
|
||||
"benchmark_name" : "simperf-test-latency",
|
||||
"common_bootbinary" : "bbl-vmlinux",
|
||||
"common_rootfs" : "poweroffnode.ext2",
|
||||
"common_outputs" : [],
|
||||
"common_simulation_outputs" : ["uartlog"]
|
||||
}
|
||||
|
|
@ -0,0 +1,31 @@
|
|||
#!/usr/bin/env bash
|
||||
|
||||
# run the bw test using the manager. optionally passing "withlaunch" will also
|
||||
# automatically launch the appropriate runfarm
|
||||
#
|
||||
# the runfarm WILL NOT be terminated upon completion
|
||||
|
||||
trap "exit" INT
|
||||
set -e
|
||||
set -o pipefail
|
||||
|
||||
#if [ "$1" == "withlaunch" ]; then
|
||||
# firesim launchrunfarm -c workloads/unittest/flash-stress-config.ini
|
||||
#fi
|
||||
|
||||
COUNTER=1
|
||||
|
||||
echo "start at" >> STRESSRUNS
|
||||
date >> STRESSRUNS
|
||||
|
||||
while [ $COUNTER -gt 0 ]; do
|
||||
firesim launchrunfarm -c workloads/unittest/flash-stress-config.ini
|
||||
firesim infrasetup -c workloads/unittest/flash-stress-config.ini
|
||||
firesim runworkload -c workloads/unittest/flash-stress-config.ini
|
||||
firesim terminaterunfarm -c workloads/unittest/flash-stress-config.ini --forceterminate
|
||||
echo "done $COUNTER"
|
||||
echo "done $COUNTER" >> STRESSRUNS
|
||||
date >> STRESSRUNS
|
||||
let COUNTER=COUNTER+1
|
||||
done
|
||||
|
||||
|
|
@ -1,42 +1,42 @@
|
|||
Debugging Using FPGA Integrated Logic Analyzers (ILA)
|
||||
=====================================================
|
||||
|
||||
Sometimes it takes too long to simulate FireSim on RTL simulators, and
|
||||
Sometimes it takes too long to simulate FireSim on RTL simulators, and
|
||||
in some occasions we would also like to debug the simulation infrastructure
|
||||
itself. For these purposes, we can use the Xilinx Integrated Logic Analyzer
|
||||
resources on the FPGA.
|
||||
resources on the FPGA.
|
||||
|
||||
ILAs allows real time sampling of pre-selected signals during FPGA runtime,
|
||||
ILAs allows real time sampling of pre-selected signals during FPGA runtime,
|
||||
and provided and interface for setting trigger and viewing samples waveforms
|
||||
from the FPGA. For more information about ILAs, please refer to the Xilinx
|
||||
guide on the topic
|
||||
guide on the topic.
|
||||
|
||||
Midas provides custom Chisel annotations which allow annotating signals in the
|
||||
Chisel source code, which will automatically generate custom ILA IP for the
|
||||
fpga, and then transforme and wire the relevant signals to the ILA.
|
||||
|
||||
ILAs consume FPGA resources, and therefore it is recommended not to annotate a
|
||||
large number of signals.
|
||||
MIDAS, in its ``targetutils`` package, provides annotations for labeling
|
||||
signals directly in the Chisel source. These will be consumed by a downstream
|
||||
FIRRTL pass which wires out the annotated signals, and binds them to an
|
||||
appropriately sized ILA instance.
|
||||
|
||||
Annotating Signals
|
||||
------------------------
|
||||
|
||||
In order to annotate a signal, we must import ``midas.passes.FpgaDebugAnnotation``.
|
||||
We then simply add a relevant ``FpgaDebugAnnotation(<selected_signal>)`` with the
|
||||
desired signal as an argument.
|
||||
|
||||
Example:
|
||||
In order to annotate a signal, we must import the
|
||||
``midas.targetutils.FpgaDebug`` annotator. FpgaDebug's apply method accepts a
|
||||
vararg of chisel3.Data. Invoke it as follows:
|
||||
|
||||
::
|
||||
|
||||
import midas.passes.FpgaDebugAnnotation
|
||||
import midas.targetutils.FpgaDebug
|
||||
|
||||
class SomeModuleIO(implicit p: Parameters) extends SomeIO()(p){
|
||||
val out1 = Output(Bool())
|
||||
val in1 = Input(Bool())
|
||||
chisel3.experimental.annotate(FpgaDebugAnnotation(out1))
|
||||
FpgaDebug(out1, in1)
|
||||
}
|
||||
|
||||
You can annotate signals throughout FireSim, including in MIDAS and
|
||||
Rocket-Chip Chisel sources, with the only exception being the Chisel3 sources
|
||||
themselves (eg. in Chisel3.util.Queue).
|
||||
|
||||
Note: In case the module with the annotated signal is instantiated multiple times,
|
||||
all instatiations of the annotated signal will be wired to the ILA.
|
||||
|
||||
|
|
@ -68,7 +68,7 @@ Follow the instructions in the `AWS-FPGA guide for connecting xilinx hardware ma
|
|||
|
||||
where ``<hostname or IP address>`` is the internal IP of the simulation instance (not
|
||||
the manager instance. i.e. The IP starting with 192.168.X.X).
|
||||
The probes file can be found in the manager instance under the path
|
||||
The probes file can be found in the manager instance under the path
|
||||
``firesim/deploy/results-build/<build_identifier>/cl_firesim/build/checkpoints/<probes_file.ltx>``
|
||||
|
||||
Select the ILA with the description of `WRAPPER_INST/CL/CL_FIRESIM_DEBUG_WIRING_TRANSFORM`, and you may now use the ILA just as if it was on
|
||||
|
|
|
|||
|
|
@ -22,26 +22,37 @@ the design/abstraction hierarchy. Ordered from least to most detailed, they are:
|
|||
simulation flow provided by AWS. Supported simulators: VCS, Vivado XSIM.
|
||||
|
||||
|
||||
Generally, MIDAS-level simulations are only slightly slower than simulating at
|
||||
target-RTL. Moving to FPGA-Level is very expensive. This illustrated in the
|
||||
chart below.
|
||||
Generally, MIDAS-level simulations are only slightly slower than target-level
|
||||
ones. Moving to FPGA-Level is very expensive. This illustrated in the chart
|
||||
below.
|
||||
|
||||
====== ===== ======= ========= =======
|
||||
Level Waves VCS Verilator XSIM
|
||||
====== ===== ======= ========= =======
|
||||
Target Off 4.8 kHz 6.2 kHz N/A
|
||||
Target On 0.8 kHz 4.8 kHz N/A
|
||||
MIDAS Off 3.8 kHz 2.0 kHz N/A
|
||||
MIDAS On 2.9 kHz 1.0 kHz N/A
|
||||
FPGA On 2.3 Hz N/A 0.56 Hz
|
||||
====== ===== ======= ========= =======
|
||||
====== ===== ======= ========= ============= ============= =======
|
||||
Level Waves VCS Verilator Verilator -O1 Verilator -O2 XSIM
|
||||
====== ===== ======= ========= ============= ============= =======
|
||||
Target Off 4.8 kHz 3.9 kHz 6.6 kHz N/A N/A
|
||||
Target On 0.8 kHz 3.0 kHz 5.1 kHz N/A N/A
|
||||
MIDAS Off 3.8 kHz 2.4 kHz 4.5 kHz 5.3 KHz N/A
|
||||
MIDAS On 2.9 kHz 1.5 kHz 2.7 kHz 3.4 KHz N/A
|
||||
FPGA On 2.3 Hz N/A N/A N/A 0.56 Hz
|
||||
====== ===== ======= ========= ============= ============= =======
|
||||
|
||||
Notes: Default configurations of a single-core Rocket Chip instance running
|
||||
rv64ui-v-add. Frequencies are given in target-Hz. Presently, the default
|
||||
Note that using more agressive optimization levels when compiling the
|
||||
Verilated-design dramatically lengths compile time:
|
||||
|
||||
====== ===== ======= ========= ============= =============
|
||||
Level Waves VCS Verilator Verilator -O1 Verilator -O2
|
||||
====== ===== ======= ========= ============= =============
|
||||
MIDAS Off 35s 48s 3m32s 4m35s
|
||||
MIDAS On 35s 49s 5m27s 6m33s
|
||||
====== ===== ======= ========= ============= =============
|
||||
|
||||
Notes: Default configurations of a single-core, Rocket-based instance running
|
||||
rv64ui-v-add. Frequencies are given in target-Hz. Presently, the default
|
||||
compiler flags passed to Verilator and VCS differ from level to level. Hence,
|
||||
these numbers are only intended to ball park simulation speeds with FireSim's
|
||||
out-of-the-box settings, not provide a scientific comparison between
|
||||
simulators.
|
||||
these numbers are only intended to ball park simulation speeds, not provide a
|
||||
scientific comparison between simulators. VCS numbers collected on Millenium,
|
||||
Verilator numbers collected on a c4.4xlarge. (ML verilator version: 4.002, TL
|
||||
verilator version: 3.904)
|
||||
|
||||
Target-Level Simulation
|
||||
--------------------------
|
||||
|
|
@ -102,14 +113,21 @@ Run all RISCV-tools assembly and benchmark tests on a verilated simulator.
|
|||
make DESIGN=FireSimNoNIC
|
||||
make DESIGN=FireSimNoNIC -j run-asm-tests
|
||||
make DESIGN=FireSimNoNIC -j run-bmark-tests
|
||||
|
||||
Run all RISCV-tools assembly and benchmark tests on a verilated simulator with waveform dumping.
|
||||
|
||||
::
|
||||
|
||||
make DESIGN=FireSimNoNIC verilator-debug
|
||||
make DESIGN=FireSimNoNIC -j run-asm-tests-debug
|
||||
make DESIGN=FireSimNoNIC -j run-bmark-tests-debug
|
||||
|
||||
Run rv64ui-p-simple (a single assembly test) on a verilated simulator.
|
||||
|
||||
::
|
||||
|
||||
make DESIGN=FireSimNoNIC
|
||||
make $(pwd)/output/f1/FireSimNoNIC-FireSimRocketChipConfig-FireSimConfig/rv64ui-p-simple.out
|
||||
make DESIGN=FireSimNoNIC $(pwd)/output/f1/FireSimNoNIC-FireSimRocketChipConfig-FireSimConfig/rv64ui-p-simple.out
|
||||
|
||||
Run rv64ui-p-simple (a single assembly test) on a VCS simulator with waveform dumping.
|
||||
|
||||
|
|
@ -117,7 +135,7 @@ Run rv64ui-p-simple (a single assembly test) on a VCS simulator with waveform du
|
|||
|
||||
|
||||
make DESIGN=FireSimNoNIC vcs-debug
|
||||
make EMUL=vcs $(pwd)/output/f1/FireSimNoNIC-FireSimRocketChipConfig-FireSimConfig/rv64ui-p-simple.vpd
|
||||
make DESIGN=FireSimNoNIC EMUL=vcs $(pwd)/output/f1/FireSimNoNIC-FireSimRocketChipConfig-FireSimConfig/rv64ui-p-simple.vpd
|
||||
|
||||
|
||||
FPGA-Level Simulation
|
||||
|
|
@ -155,6 +173,12 @@ To run a simulation you need to make both the DUT and driver targets by typing:
|
|||
make run-xsim SIM_BINARY=<PATH/TO/BINARY/FOR/TARGET/TO/RUN> # Launch the driver
|
||||
|
||||
|
||||
When following this process, you should wait until ``make xsim-dut`` prints
|
||||
``opening driver to xsim`` before running ``make run-xsim`` (getting these prints from
|
||||
``make xsim-dut`` will take a while). Additionally, you will want to use
|
||||
``DESIGN=FireSimNoNIC``, since the XSim scripts included with ``aws-fpga`` do
|
||||
not support DMA PCIS.
|
||||
|
||||
Once both processes are running, you should see:
|
||||
|
||||
::
|
||||
|
|
|
|||
|
|
@ -12,3 +12,4 @@ This section describes methods of debugging the target design and the simulation
|
|||
Debugging-Hardware-Using-ILA.rst
|
||||
TracerV.rst
|
||||
DESSERT.rst
|
||||
printf-synthesis.rst
|
||||
|
|
|
|||
|
|
@ -0,0 +1,71 @@
|
|||
Printf Synthesis
|
||||
===================
|
||||
|
||||
MIDAS can synthesize printfs present in FIRRTL (implemented as ``printf``
|
||||
statements) that would otherwise be lost in the FPGA synthesis flow. Rocket and
|
||||
BOOM have printfs of their commit logs and other useful transaction
|
||||
streams.
|
||||
|
||||
::
|
||||
|
||||
C0: 409 [1] pc=[008000004c] W[r10=0000000000000000][1] R[r 0=0000000000000000] R[r20=0000000000000003] inst=[f1402573] csrr a0, mhartid
|
||||
C0: 410 [0] pc=[008000004c] W[r 0=0000000000000000][0] R[r 0=0000000000000000] R[r20=0000000000000003] inst=[f1402573] csrr a0, mhartid
|
||||
C0: 411 [0] pc=[008000004c] W[r 0=0000000000000000][0] R[r 0=0000000000000000] R[r20=0000000000000003] inst=[f1402573] csrr a0, mhartid
|
||||
C0: 412 [1] pc=[0080000050] W[r 0=0000000000000000][0] R[r10=0000000000000000] R[r 0=0000000000000000] inst=[00051063] bnez a0, pc + 0
|
||||
C0: 413 [1] pc=[0080000054] W[r 5=0000000080000054][1] R[r 0=0000000000000000] R[r 0=0000000000000000] inst=[00000297] auipc t0, 0x0
|
||||
C0: 414 [1] pc=[0080000058] W[r 5=0000000080000064][1] R[r 5=0000000080000054] R[r16=0000000000000003] inst=[01028293] addi t0, t0, 16
|
||||
C0: 415 [1] pc=[008000005c] W[r 0=0000000000010000][1] R[r 5=0000000080000064] R[r 5=0000000080000064] inst=[30529073] csrw mtvec, t0
|
||||
|
||||
Synthesizing these printfs lets you capture the same logs on a running FireSim instance.
|
||||
|
||||
Enabling Printf Synthesis
|
||||
----------------------------
|
||||
|
||||
To synthesize a printf, in your Chisel source you need to annotate the specific
|
||||
printfs you'd like to capture. Presently, due to a limitation in Chisel and
|
||||
FIRRTL's annotation system, you need to annotate the arguments to the printf, not the printf itself,
|
||||
like so:
|
||||
|
||||
::
|
||||
|
||||
printf(midas.targetutils.SynthesizePrintf("x%d p%d 0x%x\n", rf_waddr, rf_waddr, rf_wdata))
|
||||
|
||||
Be judicious, as synthesizing many, frequently active printfs, will slow down your simulator.
|
||||
|
||||
Once your printfs have been annotated, to enable printf synthesis add the ``WithPrintfSynthesis`` Config to your
|
||||
PLATFORM_CONFIG in SimConfigs.scala. During compilation, MIDAS will print the
|
||||
number of printfs it's synthesized. In the target's generated header
|
||||
(``<DESIGN>-const.h``), you'll find metadata for each of the printfs MIDAS synthesized.
|
||||
This is passed as argument to the constructor of the ``synthesized_prints_t``
|
||||
endpoint driver, which will be automatically instantiated in FireSim driver.
|
||||
|
||||
Runtime Arguments
|
||||
-----------------
|
||||
**+print-file**
|
||||
Specifies the file into which the synthesized printf log should written.
|
||||
|
||||
**+print-start**
|
||||
Specifies the target-cycle at which the printf trace should be captured in the
|
||||
simulator. Since capturing high-bandwidth printf traces will slow down
|
||||
simulation, this allows the user to reach the region-of-interest at full simulation speed.
|
||||
|
||||
**+print-end**
|
||||
Specifies the target cycle at which to stop pulling the synthesized print
|
||||
trace from the simulator.
|
||||
|
||||
**+print-binary**
|
||||
By default, a captured printf trace will be written to file formatted
|
||||
as it would be emitted by a software RTL simulator. Setting this dumps the
|
||||
raw binary coming off the FPGA instead, improving simulation rate.
|
||||
|
||||
**+print-no-cycle-prefix**
|
||||
(Formatted output only) This removes the cycle prefix from each printf to
|
||||
save bandwidth in cases where the printf already includes a cycle field. In
|
||||
binary-output mode, since the target cycle is implicit in the token stream,
|
||||
this flag has no effect.
|
||||
|
||||
Related Publications
|
||||
--------------------
|
||||
|
||||
Printf synthesis was first presented in our FPL2018 paper, `DESSERT
|
||||
<https://people.eecs.berkeley.edu/~biancolin/papers/dessert-fpl18.pdf>`_.
|
||||
|
|
@ -0,0 +1,116 @@
|
|||
.. _firemarshal-commands:
|
||||
|
||||
FireMarshal Commands
|
||||
=======================
|
||||
|
||||
.. attention::
|
||||
|
||||
FireMarshal is still in alpha. You are encouraged to try it out and use it
|
||||
for new workloads. The old-style workload generation is still supported (see
|
||||
:ref:`defining-custom-workloads` for details).
|
||||
|
||||
|
||||
Core Options
|
||||
--------------------
|
||||
The base ``marshal`` command provides a number of options that apply to most
|
||||
sub-commands. You can also run ``marshal -h`` for the most up-to-date
|
||||
documentation.
|
||||
|
||||
``--workdir``
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
By default, FireMarshal will search the same directory as the provided
|
||||
configuration file for ``base`` references and the workload source directory.
|
||||
This option instructs FireMarshal to look elsewhere for these references.
|
||||
|
||||
``-i --initramfs``
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
By default, FireMarshal assumes that your workload includes both a rootfs and a
|
||||
boot-binary. However, it may be necessary (e.g. when using spike) to build the
|
||||
rootfs into the boot-binary and load it into RAM during boot. This is only
|
||||
supported on linux-based workloads. This option instructs FireMarshal too use
|
||||
the \*-initramfs boot-binary instead of the disk-based outputs.
|
||||
|
||||
``-v --verbose``
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
FireMarshal will redirect much of it's output to a log file in order to keep
|
||||
standard out clean. This option instructs FireMarshal to print much more output to
|
||||
standard out (in addition to logging it).
|
||||
|
||||
build
|
||||
--------------------------------------
|
||||
The build command is used to generate the rootfs's and boot-binaries from the
|
||||
workload configuration file. The output will be ``images/NAME-JOBNAME-bin`` and
|
||||
``images/NAME-JOBNAME.img`` files for each job in the workload. If you passed
|
||||
the --initramfs option to FireMarshal, a ``images/NAME-JOBNAME-bin-initramfs``
|
||||
file will also be created.
|
||||
|
||||
::
|
||||
|
||||
./marshal build [-B] [-I] config [config]
|
||||
|
||||
You may provide multiple config files to build at once.
|
||||
|
||||
``-I -B``
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
These options allow you to build only the image (rootfs) or boot-binary
|
||||
(respectively). This is occasionally useful if you have incomplete changes in
|
||||
the image or binary definitions but would still like to test the other.
|
||||
|
||||
launch
|
||||
--------------------------------------
|
||||
The launch command will run the workload in either Qemu (a high-performance
|
||||
functional simulator) or spike (the official RISC-V ISA simulator). Qemu will
|
||||
be used by default and is the best choice in most circumstances.
|
||||
|
||||
::
|
||||
|
||||
./marshal launch [-s] [-j [JOB]] config
|
||||
|
||||
``-j --job``
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
FireMarshal currently only supports launching one node at a time. By default,
|
||||
only the main workload will be run, you can specify jobs (using the job 'name')
|
||||
to run using the --job option.
|
||||
|
||||
``-s --spike``
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
In some cases, you may need to boot your workload in spike (typically due to a
|
||||
custom ISA extension or hardware model). In that case, you may use the -s
|
||||
option. Note that spike currently does not support network or block devices.
|
||||
You must pass the --initramfs option to FireMarshal when using spike.
|
||||
|
||||
clean
|
||||
--------------------------------------
|
||||
Deletes all outputs for the provided configuration (rootfs and bootbinary).
|
||||
Running the build command multiple times will re-run guest-init scripts and
|
||||
re-apply any files, but will not re-produce the base image. If you need to
|
||||
inherit changes from an updated base config, or generate a clean image (e.g. if
|
||||
the filesystem was corrupted), you must clean first.
|
||||
|
||||
test
|
||||
--------------------------------------
|
||||
The test command will build and run the workload, and compare its output
|
||||
against the ``testing`` specification provided in its configuration. See
|
||||
:ref:`firemarshal-config` for details of the testing specification. If jobs
|
||||
are specified, all jobs will be run independently and their outputs will be
|
||||
included in the output directory.
|
||||
|
||||
``-s --spike``
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
Test using spike instead of qemu (requires the --initramfs option to the
|
||||
``marshal`` command).
|
||||
|
||||
``-m testDir --manual testDir``
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
Do not build and launch the workload, simply compare it's ``testing``
|
||||
specification against a pre-existing output. This allows you to check the
|
||||
output of firesim runs against a workload. It is also useful when developing a
|
||||
workload test.
|
||||
|
||||
install
|
||||
--------------------------------------
|
||||
.. _firemarshal-install:
|
||||
|
||||
Creates a firesim workload definition file in ``firesim/deploy/workloads`` with
|
||||
all appropriate links to the generated workload. This allows you to launch the
|
||||
workload in firesim using standard commands (see :ref:`running_simulations`).
|
||||
|
|
@ -0,0 +1,326 @@
|
|||
.. _firemarshal-config:
|
||||
|
||||
Workload Specification
|
||||
=================================
|
||||
|
||||
.. attention::
|
||||
|
||||
FireMarshal is still in alpha. You are encouraged to try it out and use it
|
||||
for new workloads. The old-style workload generation is still supported (see
|
||||
:ref:`defining-custom-workloads` for details).
|
||||
|
||||
Workloads are defined by a configuration file and corresponding workload source
|
||||
directory, both typically in the ``firesim/sw/firesim-software/workloads/``
|
||||
directory. Most paths in the configuration file are assumed to be relative to
|
||||
the workload source directory.
|
||||
|
||||
Example Configuration File
|
||||
-----------------------------
|
||||
FireMarshal supports many configuration options (detailed below), many of which
|
||||
are not commonly used. We will now walk through an example that uses most of
|
||||
the common options: ``workloads/example-fed.json``. In this example, we produce
|
||||
a 2-node workload that runs two benchmarks: quicksort and spam-filtering. This
|
||||
will require installing a number of packages on Fedora, as well as
|
||||
cross-compiling some code. The configuration is as follows:
|
||||
|
||||
.. include:: example-fed.json
|
||||
:code: json
|
||||
|
||||
The ``name`` field is required and (by convention) should match the name of the
|
||||
configuration file. Next is the ``base`` (fedora-base.json). This option
|
||||
specifies an existing workload to base off of. FireMarshal will first build
|
||||
``fedora-base.json``, and use a copy of its rootfs for example-fed before
|
||||
applying the remaining options. Additionally, if fedora-base.json specifies any
|
||||
configuration options that we do not include, we will inherit those (e.g. we
|
||||
will use the ``linux-config`` option specified by fedora-base). Notice that we
|
||||
do not specify a workload source directory. FireMarshal will look in
|
||||
``workloads/example-fed/`` for any sources specified in the remaining options.
|
||||
|
||||
Next come a few options that specify common setup options used by all jobs in
|
||||
this workload. The ``overlay`` option specifies a filesystem overlay to copy
|
||||
into our rootfs. In this case, it includes the source code for our benchmarks
|
||||
(see ``workloads/example-fed/overlay``). Next is a ``host-init`` option, this
|
||||
is a script that should be run on the host before building. In our case, it
|
||||
cross-compiles the quicksort benchmark (cross-compilation is much faster than
|
||||
natively compiling).
|
||||
|
||||
.. include:: example-fed/host-init.sh
|
||||
:code: bash
|
||||
|
||||
Next is ``guest-init``, this script should run exactly once natively within our
|
||||
workload. For example-fed, this script installs a number of packages that are
|
||||
required by our benchmarks. Note that guest-init scripts are run during the
|
||||
build process; this can take a long time, especially with fedora. You will see
|
||||
linux boot messages and may even see a login prompt. There is no need to login
|
||||
or interact at all, the guest-init script will run in the background. Note that
|
||||
guest-init.sh ends with a ``poweroff`` command, all guest-init scripts should
|
||||
include this (leave it off to debug the build process).
|
||||
|
||||
.. include:: example-fed/guest-init.sh
|
||||
:code: bash
|
||||
|
||||
Finally, we specify the two jobs that will run on each simulated node. Job
|
||||
descriptions have the same format and options as normal workloads. However,
|
||||
notice that the job descriptions are much shorter than the basic descriptions.
|
||||
Jobs implicitly inherit from the root configuration. In this case, both qsort
|
||||
and spamBench will have the overlay and host/guest-init scripts already set up
|
||||
for them. If needed, you could override these options with a different ``base``
|
||||
option in the job description. In our case, we need only provide a custom
|
||||
``run`` option to each workload. The run option specifies a script that should
|
||||
run natively in each job every time the job is launched. In our case, we run
|
||||
each benchmark, collecting some statistics along the way, and then shutdown.
|
||||
Finishing a run script with ``poweroff`` is a common pattern that allows
|
||||
workloads to run automatically (no need to log-in or interact at all).
|
||||
|
||||
.. include:: example-fed/runQsort.sh
|
||||
:code: bash
|
||||
|
||||
We can now build and launch this workload:
|
||||
|
||||
::
|
||||
|
||||
./marshal build workloads/example-fed.json
|
||||
./marshal launch -j qsort workloads/example-fed.json
|
||||
./marshal launch -j spamBench workloads/example-fed.json
|
||||
|
||||
For more examples, see the ``test/`` directory that contains many workloads
|
||||
used for testing FireMarshal.
|
||||
|
||||
Bare-Metal Workloads
|
||||
-------------------------
|
||||
FireMarshal was primarily designed to support linux-based workloads. However,
|
||||
it provides basic support for bare-metal workloads. Take ``test/bare.json`` as
|
||||
an example:
|
||||
|
||||
.. include:: bare.json
|
||||
:code: json
|
||||
|
||||
This workload creates a simple "Hello World" bare-metal workload. This workload
|
||||
simply inherits from the "bare" distro in its ``base`` option. This tells
|
||||
FireMarshal to not attempt to build any linux binaries or rootfs's for this
|
||||
workload. It then includes a simple host-init script that simply calls the
|
||||
makefile to build the bare-metal boot-binary. Finally, it hard-codes a path to
|
||||
the generated boot-binary. Note that we can still use all the standard
|
||||
FireMarshal commands with bare-metal workloads. In this case, we provide a
|
||||
testing specification that simply compares the serial port output against the
|
||||
known good output of "Hello World!".
|
||||
|
||||
A complete discussion of generating bare-metal boot-binaries is out of scope
|
||||
for this documentation.
|
||||
|
||||
Configuration File Options
|
||||
----------------------------
|
||||
Below is a complete list of configuration options available to FireMarshal.
|
||||
|
||||
name
|
||||
^^^^^^^^^
|
||||
Name to use for the workload. Derived objects (rootfs/bootbin) will be named
|
||||
according to this option.
|
||||
|
||||
*Non-heritable*
|
||||
|
||||
base
|
||||
^^^^^^^^^^
|
||||
Configuration file to inherit from. FireMarshal will look in the same directory
|
||||
as the workload config file for the base configuration (or the workdir if
|
||||
``--workdir`` was passed to the marshal command). A copy of the rootfs from ``base``
|
||||
will be used when building this workload. Additionally, most configuration
|
||||
options will be inherited if not explicitly provided (options that cannot be
|
||||
inherited will be marked as 'non-heritable' in this documentation).
|
||||
|
||||
In addition to normal configuration files, you may inherit from several
|
||||
hard-coded "distros" including: fedora, br (buildroot), and bare. This is not
|
||||
recommended for the linux-based distros because the fedora-base.json and
|
||||
br-base.json configurations include useful additions to get things like serial
|
||||
ports or the network to work. However, basing on the 'bare' distro is the
|
||||
recommended way to generate bare-metal workloads.
|
||||
|
||||
*Non-heritable*
|
||||
|
||||
spike
|
||||
^^^^^^^^^^
|
||||
Path to binary for spike (riscv-isa-sim) to use when running this
|
||||
workload in spike. Useful for custom forks of spike to support custom
|
||||
instructions or hardware models. Defaults to the version of spike on your PATH
|
||||
(typically the one include with riscv-tools).
|
||||
|
||||
linux-src
|
||||
^^^^^^^^^^^^^^^^
|
||||
Path to riscv-linux source directory to use when building the boot-binary for
|
||||
this workload. Defaults to the riscv-linux source submoduled at
|
||||
``firesim/sw/firesim-software/riscv-linux``.
|
||||
|
||||
linux-config
|
||||
^^^^^^^^^^^^^^^^
|
||||
Linux configuration file to use when building linux. Take care when using a
|
||||
custom configuration, FireSim may require certain boot arguments and device
|
||||
drivers to work properly.
|
||||
|
||||
host-init
|
||||
^^^^^^^^^^^^^^
|
||||
A script to run natively on your host (i.e., your manager instance where you
|
||||
invoked FireMarshal) from the workload source directory each time you
|
||||
explicitly build this workload.
|
||||
|
||||
*Non-heritable*
|
||||
|
||||
guest-init
|
||||
^^^^^^^^^^^^^^^ A script to run natively on the guest (i.e., your workload
|
||||
running in qemu) exactly once while building. The guest init script will be run
|
||||
from the root directory with root privileges. This script should end with a
|
||||
call to ``poweroff`` to make the build process fully automated. Otherwise, the
|
||||
user will need to log in and shut down manually on each build.
|
||||
|
||||
post_run_hook
|
||||
^^^^^^^^^^^^^^^^^
|
||||
A script or command to run on the output of your run. At least the serial port output of
|
||||
each run is captured, along with any file outputs specified in the ``outputs``
|
||||
option. The script will be called like so:
|
||||
|
||||
::
|
||||
|
||||
cd workload-dir
|
||||
post_run_hook /path/to/output
|
||||
|
||||
The output directory will follow roughly the following format:
|
||||
|
||||
::
|
||||
|
||||
runOutput/name-DATETIME-RAND/
|
||||
name-job/
|
||||
uartlog
|
||||
OUTPUT_FILE1
|
||||
...
|
||||
OUTPUT_FILEN
|
||||
|
||||
When running as part of the ``test`` command, there will be a folder for each
|
||||
job in the workload.
|
||||
|
||||
overlay
|
||||
^^^^^^^^^^^^
|
||||
Filesystem overlay to apply to the workload rootfs. An overlay should match the
|
||||
rootfs directory structure, with the overlay directory corresponding to the
|
||||
root directory. This is especially useful for overriding system configuration
|
||||
files (e.g. /etc/fstab). The owner of all copied files will be changed to root
|
||||
in the workload rootfs after copying.
|
||||
|
||||
files
|
||||
^^^^^^^^^^
|
||||
A list of files to copy into the rootfs. The file list has the following format:
|
||||
|
||||
::
|
||||
|
||||
[ ["src1", "dst1"], ["src2", "dst2"], ... ]
|
||||
|
||||
The source paths are relative to the workload source directory, the destination
|
||||
paths are absolute with respect to the workload rootfs (e.g. ["file1",
|
||||
"/root/"]). The ownership of each file will be changed to 'root' after copying.
|
||||
|
||||
outputs
|
||||
^^^^^^^^^^^^
|
||||
A list of files to copy out of the workload rootfs after running. Each path
|
||||
should be absolute with respect to the workload rootfs. Files will be placed
|
||||
together in the output directory. You cannot specify the directory structure of
|
||||
the output.
|
||||
|
||||
run
|
||||
^^^^^^^^^^^^^
|
||||
A script to run automatically every time this workload runs. The script will
|
||||
run after all other initialization finishes, but does not require the user to
|
||||
log in (run scripts run concurrently with any user interaction). Run scripts
|
||||
typically end with a call to ``poweroff`` to make the workload fully automated,
|
||||
but this can be omitted if you would like to interact with the workload after
|
||||
its run script has finished.
|
||||
|
||||
.. Note:: Unlike FireSim workloads, the FireMarshal launch command uses
|
||||
the same rootfs for each run (not a copy), so you should avoid using ``poweroff
|
||||
-f`` to prevent filesystem corruption.
|
||||
|
||||
*Non-heritable*
|
||||
|
||||
command
|
||||
^^^^^^^^^^^^^
|
||||
A command to run every time this workload runs. The command will be run from
|
||||
the root directory and will automatically call ``poweroff`` when complete (the
|
||||
user does not need to include this).
|
||||
|
||||
*Non-heritable*
|
||||
|
||||
workdir
|
||||
^^^^^^^^^^^
|
||||
Directory to use as the workload source directory. Defaults to a directory with
|
||||
the same name as the configuration file.
|
||||
|
||||
*Non-heritable*
|
||||
|
||||
launch
|
||||
^^^^^^^^^^^
|
||||
Enable/Disable launching of a job when running the 'test' command. This is
|
||||
occasionally needed for special 'dummy' workloads or other special-purpose jobs
|
||||
that only make sense when running on FireSim. Defaults to 'yes'.
|
||||
|
||||
jobs
|
||||
^^^^^^^^^
|
||||
A list of configurations describing individual jobs that make up this workload.
|
||||
This list is ordered (FireSim places these jobs in-order in simulation slots).
|
||||
Job descriptions have the same syntax and options as normal workloads. The one
|
||||
exception is that jobs implicitly inherit from the parent workload unless a
|
||||
``base`` option is explicitly provided. The job name will be appended to the
|
||||
workload name when creating boot-binaries and rootfs's. For example, a workload
|
||||
called "foo" with two jobs named 'bar' and 'baz' would create 3 rootfs's:
|
||||
foo.img, foo-bar.img, and foo-baz.img.
|
||||
|
||||
*Non-heritable*: You cannot use jobs as a ``base``, only base workloads.
|
||||
|
||||
bin
|
||||
^^^^^^^^^
|
||||
Explicit path to the boot-binary to use. This will override any generated
|
||||
binaries created during the build process. This is particularly useful for
|
||||
bare-metal workloads that generate their own raw boot code.
|
||||
|
||||
*Non-heritable*
|
||||
|
||||
img
|
||||
^^^^^^^^^
|
||||
Explicit path to the rootfs to use. This will override any generated rootfs
|
||||
created during the build process. This is mostly used for debugging.
|
||||
|
||||
*Non-heritable*
|
||||
|
||||
testing
|
||||
^^^^^^^^^^^^^
|
||||
Provide details of how to test this workload. The ``test`` command will ignore
|
||||
any workload that does not have a ``testing`` field. This option is a map with
|
||||
the following options (only ``refDir`` is required):
|
||||
|
||||
*Non-heritable*
|
||||
|
||||
refDir
|
||||
""""""""""""""
|
||||
Path to a directory containing reference outputs for this workload. Directory
|
||||
structures are compared directly (same folders, same file names). Regular files
|
||||
are compared exactly. Serial outputs (uartlog) need only match a subset of
|
||||
outputs; the entire reference uartlog contents must exist somewhere
|
||||
(contiguously) in the test uartlog.
|
||||
|
||||
buildTimeout
|
||||
""""""""""""""""""""
|
||||
Maximum time (in seconds) that the workload should take to build. The test will
|
||||
fail if building takes longer than this. Defaults to infinite.
|
||||
|
||||
.. Note:: workloads with many jobs and guest-init scripts, could take a very
|
||||
long time to build.
|
||||
|
||||
runTimeout
|
||||
""""""""""""""""
|
||||
Maximum time (in seconds) that any particular job should take to run and exit.
|
||||
The test will fail if a job runs for longer than this before exiting. Defaults
|
||||
to infinite.
|
||||
|
||||
strip
|
||||
"""""""""""""
|
||||
Attempt to clean up the uartlog output before comparing against the reference.
|
||||
This will remove all lines not generated by a run script or command, as well as
|
||||
stripping out any extra characters that might be added by the run-system (e.g.
|
||||
the systemd timestamps on Fedora). This option is highly recommended on Fedora
|
||||
due to it's non-deterministic output.
|
||||
|
|
@ -0,0 +1,70 @@
|
|||
Quick Start
|
||||
--------------------------------------
|
||||
|
||||
.. attention::
|
||||
|
||||
FireMarshal is still in alpha. You are encouraged to try it out and use it
|
||||
for new workloads. The old-style workload generation is still supported (see
|
||||
:ref:`defining-custom-workloads` for details).
|
||||
|
||||
|
||||
All workload-generation related commands and code are in ``firesim/sw/firesim-software``.
|
||||
|
||||
FireMarshal comes with a few basic workloads that you can build right out of
|
||||
the box (in ``workloads/``). In this example, we will build and test the
|
||||
buildroot-based linux distribution (called *br-base*). We begin by building the
|
||||
workload:
|
||||
|
||||
::
|
||||
|
||||
./marshal build workloads/br-base.json
|
||||
|
||||
The first time you build a workload may take a long time (buildroot must
|
||||
download and cross-compile a large number of packages), but subsequent builds
|
||||
of the same base will use cached results. Once the command completes, you
|
||||
should see two new files in ``images/``: ``br-base-bin`` and ``br-base.img``.
|
||||
These are the boot-binary (linux + boot loader) and root filesystem
|
||||
(respectively). We can now launch this workload in qemu:
|
||||
|
||||
::
|
||||
|
||||
./marshal launch workloads/br-base.json
|
||||
|
||||
You should now see linux booting and be presented with a login prompt. Sign in
|
||||
as 'root' with password 'firesim'. From here you can manipulate files, run
|
||||
commands, and generally use the image as if it had booted on real hardware. Any
|
||||
changes you make here will be persistent between reboots. Once you are done
|
||||
exploring, simply shutdown the workload:
|
||||
|
||||
::
|
||||
|
||||
$ poweroff
|
||||
|
||||
It is typically not a good idea to modify the \*-base workloads directly since
|
||||
many other workloads might inherit those changes. To make sure that we've
|
||||
cleaned out any changes, let's clean and rebuild the workload:
|
||||
|
||||
::
|
||||
|
||||
./marshal clean workloads/br-base.json
|
||||
./marshal build workloads/br-base.json
|
||||
|
||||
Note that this build took significantly less time than the first; FireMarshal
|
||||
caches intermediate build steps whenever possible. The final step is to run
|
||||
this workload on the real firesim RTL with full timing accuracy. To do that we
|
||||
must first install the workload:
|
||||
|
||||
::
|
||||
|
||||
./marshal install workloads/br-base.json
|
||||
|
||||
This command creates a firesim workload file at
|
||||
``firesim/deploy/workloads/br-base.json``. You can now run this workload using
|
||||
the standard FireSim commands (e.g. :ref:`single-node-sim`, just change the
|
||||
``workloadname`` option to "br-base.json" from "linux-uniform.json").
|
||||
|
||||
.. attention:: While the FireMarshal ``install`` command is the recommended way to create
|
||||
firesim configurations, you can still hand-create firesim workloads if needed.
|
||||
For example, the linux-uniform workload described in :ref:`single-node-sim` is
|
||||
a manually created workload that uses the br-base-bin and br-base.img files
|
||||
directly.
|
||||
|
|
@ -0,0 +1,9 @@
|
|||
{
|
||||
"name" : "bare",
|
||||
"base" : "bare",
|
||||
"host-init" : "build.sh",
|
||||
"bin" : "hello",
|
||||
"testing" : {
|
||||
"refDir" : "refOutput"
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,17 @@
|
|||
{
|
||||
"name" : "example-fed",
|
||||
"base" : "fedora-base.json",
|
||||
"overlay" : "overlay",
|
||||
"guest-init" : "guest-init.sh",
|
||||
"host-init" : "host-init.sh",
|
||||
"jobs" : [
|
||||
{
|
||||
"name" : "qsort",
|
||||
"run" : "runQsort.sh"
|
||||
},
|
||||
{
|
||||
"name" : "spamBench",
|
||||
"run" : "runSpam.sh"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
|
@ -0,0 +1,9 @@
|
|||
#!/bin/bash
|
||||
|
||||
echo "Installing the real time tool (not the shell builtin)"
|
||||
dnf install -y time
|
||||
|
||||
echo "Installing the spambayes python module for the spam benchmark"
|
||||
pip install spambayes
|
||||
|
||||
poweroff
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
#!/bin/bash
|
||||
|
||||
echo "Building qsort benchmark"
|
||||
cd overlay/root/qsort
|
||||
|
||||
make
|
||||
|
|
@ -0,0 +1,6 @@
|
|||
#!/bin/bash
|
||||
set -x
|
||||
|
||||
cd root/qsort
|
||||
/usr/bin/time -f "%S,%M,%F" ./qsort 10000 2> ../run_result.csv
|
||||
poweroff
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
#!/bin/bash
|
||||
set -x
|
||||
|
||||
# This script will be run every time you boot the workload. In this case we're
|
||||
# running a benchmark and recording some timing information into a csv that can
|
||||
# be extracted later. Also note that we call poweroff at the end, if you would
|
||||
# prefer to interact with the workload after it's booted, you can leave that off.
|
||||
|
||||
cd root/spamBench
|
||||
/usr/bin/time -f "%S,%M,%F" ./bm_spambayes.py 2> ../run_result.csv
|
||||
poweroff
|
||||
|
|
@ -0,0 +1,34 @@
|
|||
.. _firemarshal:
|
||||
|
||||
FireMarshal (alpha)
|
||||
=======================================
|
||||
.. attention::
|
||||
|
||||
FireMarshal is still in alpha. You are encouraged to try it out and use it
|
||||
for new workloads. The old-style workload generation is still supported (see
|
||||
:ref:`defining-custom-workloads` for details).
|
||||
|
||||
Workload generation in FireSim is handled by a tool called **FireMarshal** in
|
||||
``firesim/sw/firesim-software/``.
|
||||
|
||||
**Workloads** in FireMarshal consist of a series of **Jobs** that are assigned
|
||||
to logical nodes in the target system. If no jobs are specified, then the
|
||||
workload is considered ``uniform`` and only a single image will be produced for
|
||||
all nodes in the system. Workloads are described by a json file and a
|
||||
corresponding workload directory and can inherit their definitions from
|
||||
existing workloads. Typically, workload configurations are kept in
|
||||
``workloads`` although you can use any directory you like. We provide a few
|
||||
basic workloads to start with including buildroot or Fedora-based linux
|
||||
distributions and bare-metal.
|
||||
|
||||
Once you define a workload, the ``marshal`` command will produce a
|
||||
corresponding boot-binary and rootfs for each job in the workload. This binary
|
||||
and rootfs can then be launched on qemu or spike (for functional simulation), or
|
||||
installed to firesim for running on real RTL.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
FireMarshal-QuickStart
|
||||
FireMarshal-Commands
|
||||
FireMarshal-Config
|
||||
|
|
@ -1,5 +1,5 @@
|
|||
Targets
|
||||
================
|
||||
=======
|
||||
|
||||
FireSim generates SoC models by transforming RTL emitted by a Chisel
|
||||
generator, such as the Rocket SoC generator. Subject to
|
||||
|
|
@ -21,6 +21,8 @@ transformed and thus used in FireSim:
|
|||
These are replaced with synchronously reset registers using a FIRRTL transformation.
|
||||
|
||||
|
||||
.. _generating-different-targets:
|
||||
|
||||
Generating Different Target-RTL
|
||||
---------------------------------
|
||||
|
||||
|
|
@ -31,13 +33,14 @@ resides in ``sim/``.
|
|||
|
||||
These projects are:
|
||||
|
||||
1. **firesim** (Default): Rocket-chip-based targets. These include targets with
|
||||
1. **firesim** (Default): rocket chip-based targets. These include targets with
|
||||
either BOOM or rocket pipelines, and should be your starting point if you're
|
||||
building an SoC with the Rocket-Chip generator.
|
||||
2. **midasexamples**: Contains the `MIDAS example designs
|
||||
building an SoC with the Rocket Chip generator.
|
||||
2. **midasexamples**: the `MIDAS example designs
|
||||
<https://github.com/ucb-bar/midas-examples>`_, a set of simple chisel
|
||||
circuits like GCD, that demonstrate how to use MIDAS. These are useful test
|
||||
cases for bringing up new MIDAS features.
|
||||
3. **fasedtests**: designs to do integration testing of FASED memory-system timing models.
|
||||
|
||||
Projects have the following directory structure:
|
||||
|
||||
|
|
@ -116,12 +119,13 @@ Single-core BOOM, no network interface
|
|||
make DESIGN=FireBoomNoNIC TARGET_CONFIG=FireSimBoomConfig
|
||||
|
||||
|
||||
-----------------------
|
||||
Changing The DRAM Model
|
||||
-----------------------
|
||||
----------------------------------------------------------
|
||||
Generating A Different FASED Memory-Timing Model Instance
|
||||
----------------------------------------------------------
|
||||
|
||||
MIDAS can generate a space of different DRAM model instances: we give some
|
||||
typical ones here. These targets use the Makefile-defined defaults of
|
||||
MIDAS's memory-timing model generator, FASED, can elaborate a space of
|
||||
different DRAM model instances: we give some typical ones here. These targets
|
||||
use the Makefile-defined defaults of
|
||||
``DESIGN=FireSim TARGET_CONFIG=FireSimRocketChipConfig``.
|
||||
|
||||
Quad-rank DDR3 first-come first-served memory access scheduler
|
||||
|
|
@ -137,7 +141,7 @@ Quad-rank DDR3 first-ready, first-come first-served memory access scheduler
|
|||
make PLATFORM_CONFIG=FireSimDDR3FRFCFSConfig
|
||||
|
||||
|
||||
As above, but with an 4 MiB (max capacity) last-level-cache model
|
||||
As above, but with a 4 MiB (maximum simulatable capacity) last-level-cache model
|
||||
|
||||
::
|
||||
|
||||
|
|
@ -174,3 +178,27 @@ Generate the GCD midas-example
|
|||
::
|
||||
|
||||
make DESIGN=GCD TARGET_PROJECT=midasexamples
|
||||
|
||||
FASED Tests (fasedtests project)
|
||||
--------------------------------------------------
|
||||
This project generates target designs capable of driving considerably more
|
||||
bandwidth to an AXI4-memory slave than current FireSim-targets. Used used to do
|
||||
integration and stress testing of FASED instances.
|
||||
|
||||
--------
|
||||
Examples
|
||||
--------
|
||||
|
||||
Generate a synthesizable AXI4Fuzzer (based off of Rocket Chip's TL fuzzer), driving a
|
||||
DDR3 FR-FCFS-based FASED instance.
|
||||
|
||||
::
|
||||
|
||||
make TARGET_PROJECT=midasexamples DESIGN=AXI4Fuzzer PLATFORM_CONFIG=FRFCFSConfig
|
||||
|
||||
As above, but with a fuzzer configue to drive 10 million transactions through
|
||||
the instance.
|
||||
|
||||
::
|
||||
|
||||
make TARGET_PROJECT=midasexamples DESIGN=AXI4Fuzzer PLATFORM_CONFIG=NT10e7_FRFCFSConfig
|
||||
|
|
|
|||
|
|
@ -1,7 +1,8 @@
|
|||
usage: firesim [-h] [-c RUNTIMECONFIGFILE] [-b BUILDCONFIGFILE]
|
||||
[-r BUILDRECIPESCONFIGFILE] [-a HWDBCONFIGFILE]
|
||||
[-x OVERRIDECONFIGDATA] [-f TERMINATESOMEF116]
|
||||
[-g TERMINATESOMEF12] [-m TERMINATESOMEM416] [-q]
|
||||
[-g TERMINATESOMEF12] [-i TERMINATESOMEF14]
|
||||
[-m TERMINATESOMEM416] [-q]
|
||||
|
||||
{managerinit,buildafi,launchrunfarm,infrasetup,boot,kill,terminaterunfarm,runworkload,shareagfi,runcheck}
|
||||
|
||||
|
|
@ -35,6 +36,9 @@ optional arguments:
|
|||
-g TERMINATESOMEF12, --terminatesomef12 TERMINATESOMEF12
|
||||
Only used by terminatesome. Terminates this many of
|
||||
the previously launched f1.2xlarges.
|
||||
-i TERMINATESOMEF14, --terminatesomef14 TERMINATESOMEF14
|
||||
Only used by terminatesome. Terminates this many of
|
||||
the previously launched f1.4xlarges.
|
||||
-m TERMINATESOMEM416, --terminatesomem416 TERMINATESOMEM416
|
||||
Only used by terminatesome. Terminates this many of
|
||||
the previously launched m4.16xlarges.
|
||||
|
|
|
|||
|
|
@ -40,8 +40,8 @@ you should not change it unless you are done with your current Run Farm.
|
|||
|
||||
Per AWS restrictions, this tag can be no longer than 255 characters.
|
||||
|
||||
``f1_16xlarges``, ``m4_16xlarges``, ``f1_2xlarges``
|
||||
""""""""""""""""""""""""""""""""""""""""""""""""""""
|
||||
``f1_16xlarges``, ``m4_16xlarges``, ``f1_4xlarges``, ``f1_2xlarges``
|
||||
"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
|
||||
|
||||
Set these three values respectively based on the number and types of instances
|
||||
you need. While we could automate this setting, we choose not to, so that
|
||||
|
|
@ -330,6 +330,7 @@ This specifies hardware parameters of the simulation environment - for example,
|
|||
selecting between a Latency-Bandwidth Pipe or DDR3 memory models.
|
||||
These are defined in ``firesim/sim/src/main/scala/firesim/SimConfigs.scala``.
|
||||
|
||||
|
||||
``instancetype``
|
||||
"""""""""""""""""""
|
||||
|
||||
|
|
@ -347,6 +348,16 @@ you should leave this set to ``None``. This is usually only used if you have
|
|||
proprietary RTL that you bake into an FPGA image, but don't want to share with
|
||||
users of the simulator.
|
||||
|
||||
``TARGET_PROJECT`` `(Optional)`
|
||||
"""""""""""""""""""""""""""""""
|
||||
|
||||
This specifies the target project in which the target is defined (this is described
|
||||
in greater detail :ref:`here<generating-different-targets>`). If
|
||||
``TARGET_PROJECT`` is undefined the manager will default to ``firesim``.
|
||||
Setting ``TARGET_PROJECT`` is required for building the MIDAS examples
|
||||
(``TARGET_PROJECT=midasexamples``) with the manager, or for building a
|
||||
user-provided target project.
|
||||
|
||||
.. _config-hwdb:
|
||||
|
||||
``config_hwdb.ini``
|
||||
|
|
|
|||
|
|
@ -96,7 +96,7 @@ that someone else owns and gave you access to.
|
|||
---------------------------
|
||||
|
||||
This command launches a Run Farm on which you run simulations. Run Farms
|
||||
consist of ``f1.16xlarge``, ``f1.2xlarge``, and ``m4.16xlarge`` instances.
|
||||
consist of ``f1.16xlarge``, ``f1.4xlarge``, ``f1.2xlarge``, and ``m4.16xlarge`` instances.
|
||||
Before you run the command, you define the number of each that you want in
|
||||
``config_runtime.ini``.
|
||||
|
||||
|
|
@ -155,8 +155,9 @@ RUN FARM WITHOUT PROMPTING FOR CONFIRMATION:
|
|||
|
||||
There a few additional commandline arguments that let you terminate only
|
||||
some of the instances in a particular Run Farm: ``--terminatesomef116 INT``,
|
||||
``--terminatesomef12 INT``, and ``--terminatesomem416 INT``, which will terminate
|
||||
ONLY as many of each type of instance as you specify.
|
||||
``--terminatesomef14 INT``, ``--terminatesomef12 INT``, and
|
||||
``--terminatesomem416 INT``, which will terminate ONLY as many of each type of
|
||||
instance as you specify.
|
||||
|
||||
Here are some examples:
|
||||
|
||||
|
|
|
|||
|
|
@ -20,11 +20,14 @@ To Remote Desktop into your manager instance, you must do the following:
|
|||
|
||||
::
|
||||
|
||||
curl https://s3.amazonaws.com/aws-fpga-developer-ami/1.4.0/Scripts/setup_gui.sh -o /home/centos/src/scripts/setup_gui.sh
|
||||
curl https://s3.amazonaws.com/aws-fpga-developer-ami/1.5.0/Scripts/setup_gui.sh -o /home/centos/src/scripts/setup_gui.sh
|
||||
sudo sed -i 's/enabled=0/enabled=1/g' /etc/yum.repos.d/CentOS-CR.repo
|
||||
/home/centos/src/scripts/setup_gui.sh
|
||||
# keep manager paramiko compatibility
|
||||
sudo pip2 uninstall gssapi
|
||||
|
||||
The former two commands are required due to AWS FPGA Dev AMI 1.3.5 incompatibilities. See
|
||||
|
||||
See
|
||||
|
||||
https://forums.aws.amazon.com/message.jspa?messageID=848073#848073
|
||||
|
||||
|
|
|
|||
|
|
@ -1,87 +1,140 @@
|
|||
Supernode
|
||||
===============
|
||||
Supernode - Multiple Simulated SoCs Per FPGA
|
||||
============================================
|
||||
|
||||
Supernode support is currently in beta. Supernode is designed to improve FPGA
|
||||
resource utilization for smaller designs and allow realistic rack topology
|
||||
simulation (32 simulated nodes) using a single ``f1.16xlarge`` instance. The
|
||||
supernode beta can be found on the ``supernode-beta`` branch of the FireSim
|
||||
repository. Supernode requires slight changes in build and runtime
|
||||
configurations. More details about supernode can be found in the `FireSim ISCA
|
||||
2018 Paper <https://sagark.org/assets/pubs/firesim-isca2018.pdf>`__.
|
||||
Supernode allows users to run multiple simulated SoCs per-FPGA in order to improve
|
||||
FPGA resource utilization and reduce cost. For example, in the case of using
|
||||
FireSim to simulate a datacenter scale system, supernode mode allows realistic
|
||||
rack topology simulation (32 simulated nodes) using a single ``f1.16xlarge``
|
||||
instance (8 FPGAs).
|
||||
|
||||
Intro
|
||||
-----------
|
||||
Below, we outline the build and runtime configuration changes needed to utilize
|
||||
supernode designs. Supernode is currently only enabled for RocketChip designs
|
||||
with NICs. More details about supernode can be found in the `FireSim ISCA 2018
|
||||
Paper <https://sagark.org/assets/pubs/firesim-isca2018.pdf>`__.
|
||||
|
||||
Supernode packs 4 identical designs into a single FPGA, and utilizes all 4 DDR
|
||||
channels available for each FPGA on AWS F1 instances. It currently does so by
|
||||
generating a wrapper top level target which encapsualtes the four simulated
|
||||
target nodes. The packed nodes are treated as 4 separate nodes, are assigned their
|
||||
own individual MAC addresses, and can perform any action a single node could:
|
||||
run different programs, interact with each other over the network, utilize
|
||||
different block device images, etc.
|
||||
Introduction
|
||||
--------------
|
||||
|
||||
Build
|
||||
-----------
|
||||
By default, supernode packs 4 identical designs into a single FPGA, and
|
||||
utilizes all 4 DDR channels available on each FPGA on AWS F1 instances. It
|
||||
currently does so by generating a wrapper top level target which encapsualtes
|
||||
the four simulated target nodes. The packed nodes are treated as 4 separate
|
||||
nodes, are assigned their own individual MAC addresses, and can perform any
|
||||
action a single node could: run different programs, interact with each other
|
||||
over the network, utilize different block device images, etc. In the networked
|
||||
case, 4 separate network links are presented to the switch-side.
|
||||
|
||||
The Supernode beta can be found on the ``supernode-beta`` branch of the FireSim
|
||||
repo. Here, we outline some of the changes between supernode and regular
|
||||
simulations. The Supernode target wrapper can be found in
|
||||
``firesim/sim/src/main/scala/SimConfigs.scala``. For example:
|
||||
Building Supernode Designs
|
||||
----------------------------
|
||||
|
||||
Here, we outline some of the changes between supernode and regular simulations
|
||||
that are required to build supernode designs.
|
||||
|
||||
The Supernode target configuration wrapper can be found in
|
||||
``firesim/sim/src/main/scala/firesim/TargetConfigs.scala``. An example wrapper
|
||||
configuration is:
|
||||
|
||||
::
|
||||
|
||||
class SupernodeFireSimRocketChipConfig extends Config(
|
||||
new WithNumNodes(4) ++
|
||||
new FireSimRocketChipConfig)
|
||||
class SupernodeFireSimRocketChipConfig extends Config(new WithNumNodes(4)
|
||||
++ new FireSimRocketChipConfig)
|
||||
|
||||
In this example, ``SupernodeFireSimRocketChipConfig`` is the wrapper, while
|
||||
``FireSimRocketChipConfig`` is the target node configuration. Therefore, if we
|
||||
want to simulate a different target configuration, we will generate a new
|
||||
Supernode wrapper, with the new target configuration. For example:
|
||||
``FireSimRocketChipConfig`` is the target node configuration. To simulate a
|
||||
different target configuration, we will generate a new supernode wrapper, with
|
||||
the new target configuration. For example, to simulate 4 quad-core nodes on one
|
||||
FPGA, you can use:
|
||||
|
||||
::
|
||||
|
||||
class SupernodeFireSimRocketChipQuadCoreConfig extends Config(
|
||||
new WithNumNodes(4) ++
|
||||
new FireSimRocketChipQuadCoreConfig)
|
||||
class SupernodeFireSimRocketChipQuadCoreConfig extends Config(new
|
||||
WithNumNodes(4) ++ new FireSimRocketChipQuadCoreConfig)
|
||||
|
||||
|
||||
Next, when defining the build recipe, we must remmber to use the supernode
|
||||
configuration: The ``DESIGN`` parameter should always be set to
|
||||
``SupernodeTop``, while the ``TARGET_CONFIG`` parameter should be set to the
|
||||
wrapper configuration that was defined in
|
||||
``firesim/sim/src/main/scala/SimConfigs.scala``. The ``PLATFORM_CONFIG`` can
|
||||
be selected the same as in regular FireSim configurations. For example:
|
||||
``FireSimSupernode``, while the ``TARGET_CONFIG`` parameter should be set to
|
||||
the wrapper configuration that was defined in
|
||||
``firesim/sim/src/main/scala/firesim/TargetConfigs.scala``. The
|
||||
``PLATFORM_CONFIG`` can be selected the same as in regular FireSim
|
||||
configurations. For example:
|
||||
|
||||
::
|
||||
|
||||
DESIGN=SupernodeTop
|
||||
DESIGN=FireSimSupernode
|
||||
TARGET_CONFIG=SupernodeFireSimRocketChipQuadCoreConfig
|
||||
PLATFORM_CONFIG=FireSimDDR3FRFCFSLLC4MBConfig
|
||||
PLATFORM_CONFIG=FireSimDDR3FRFCFSLLC4MBConfig90MHz
|
||||
instancetype=c4.4xlarge
|
||||
deploytriplet=None
|
||||
|
||||
|
||||
We currently do not provide pre-built AGFIs for supernode. You must build your
|
||||
own, using the supplied samples on the ``supernode-beta`` branch.
|
||||
We currently provide a single pre-built AGFI for supernode of 4 quad-core
|
||||
RocketChips with DDR3 memory models. You can build your own AGFI, using the supplied samples in
|
||||
``config_build_recipes.ini``. Importantly, in order to meet FPGA timing
|
||||
contraints, Supernode target may require lower host clock frequencies.
|
||||
host clock frequencies can be configured as parts of the PLATFORM_CONFIG in
|
||||
``config_build_recipes.ini``.
|
||||
|
||||
Running simulations
|
||||
--------------------
|
||||
Running Supernode Simulations
|
||||
-----------------------------
|
||||
|
||||
Running FireSim in supernode mode follows the same process as in
|
||||
"regular" mode. Currently, the only difference is that the standard input and
|
||||
standard output of the simulated nodes are written to files in the dispatched
|
||||
simulation directory, rather than the main simulation screen.
|
||||
"regular" mode. Currently, the only difference is that the main simulation
|
||||
screen remains with the name ``fsim0``, while the three other simulation screens
|
||||
can be accessed by attaching ``screen`` to ``uartpty1``, ``uartpty2``, ``uartpty3``
|
||||
respectively. All simulation screens will generate uart logs (``uartlog1``,
|
||||
``uartlog2``, ``uartlog3``). Notice that you must use ``sudo`` in order to
|
||||
attach to the uartpty or view the uart logs. The additional uart logs will not
|
||||
be copied back to the manager instance by default (as in a "regular" FireSim
|
||||
simulation). It is neccessary to specify the copying of the additional uartlogs
|
||||
(uartlog1, uartlog2, uartlog3) in the workload definition.
|
||||
|
||||
Here are some important pieces that you can use to run an example 32-node config
|
||||
on a single ``f1.16xlarge``. Better documentation will be available later:
|
||||
Supernode topologies utilize a ``FireSimSuperNodeServerNode`` class in order to
|
||||
represent one of the 4 simulated target nodes which also represents a single
|
||||
FPGA mapping, while using a ``FireSimDummyServerNode`` class which represent
|
||||
the other three simulated target nodes which do not represent an FPGA mapping.
|
||||
In supernode mode, topologies should always add nodes in pairs of 4, as one
|
||||
``FireSimSuperNodeServerNode`` and three ``FireSimDummyServerNode`` s.
|
||||
|
||||
- Sample runtime config: https://github.com/firesim/firesim/blob/supernode-beta/deploy/sample-backup-configs/sample_config_runtime.ini
|
||||
- Sample topology definition: https://github.com/firesim/firesim/blob/supernode-beta/deploy/runtools/user_topology.py#L33
|
||||
Various example Supernode topologies are provided, ranging from 4 simulated
|
||||
target nodes to 1024 simulated target nodes.
|
||||
|
||||
Below are a couple of useful examples as templates for writing custom
|
||||
Supernode topologies.
|
||||
|
||||
|
||||
A sample Supernode topology of 4 simulated target nodes which can fit on a
|
||||
single ``f1.2xlarge`` is:
|
||||
|
||||
::
|
||||
|
||||
def supernode_example_4config(self):
|
||||
self.roots = [FireSimSwitchNode()]
|
||||
servers = [FireSimSuperNodeServerNode()] + [FireSimDummyServerNode() for x in range(3)]
|
||||
self.roots[0].add_downlinks(servers)
|
||||
|
||||
|
||||
A sample Supernode topology of 32 simulated target nodes which can fit on a
|
||||
single ``f1.16xlarge`` is:
|
||||
|
||||
::
|
||||
|
||||
def supernode_example_32config(self):
|
||||
self.roots = [FireSimSwitchNode()]
|
||||
servers = UserTopologies.supernode_flatten([[FireSimSuperNodeServerNode(), FireSimDummyServerNode(), FireSimDummyServerNode(), FireSimDummyServerNode()] for y in range(8)])
|
||||
self.roots[0].add_downlinks(servers)
|
||||
|
||||
|
||||
Supernode ``config_runtime.ini`` requires selecting a supernode agfi in conjunction with a defined supernode topology.
|
||||
|
||||
|
||||
Work in Progress!
|
||||
--------------------
|
||||
|
||||
We are currently working on restructuring supernode support to support a
|
||||
wider-variety of use cases. More documentation will follow once we complete
|
||||
this rewrite.
|
||||
We are currently working on restructuring supernode to support a
|
||||
wider-variety of use cases (including non-networked cases, and increased
|
||||
packing of nodes). More documentation will follow.
|
||||
Not all FireSim features are currently available on Supernode. As a
|
||||
rule-of-thumb, target-related features have a higher likelihood of being
|
||||
supported "out-of-the-box", while features which involve external interfaces
|
||||
(such as TracerV) has a lesser likelihood of being supported "out-of-the-box"
|
||||
|
|
|
|||
|
|
@ -1,38 +1,58 @@
|
|||
.. _booting-fedora:
|
||||
|
||||
Running Fedora on FireSim
|
||||
===========================
|
||||
=====================================
|
||||
All workload-generation related commands and code are in ``firesim/sw/firesim-software``.
|
||||
|
||||
You can boot Fedora disk images pulled from upstream on FireSim simulations.
|
||||
These instructions assume you've already run through the tutorials.
|
||||
|
||||
|
||||
Building a FireSim-compatible Fedora Image
|
||||
-----------------------------------------------
|
||||
|
||||
To download and build a Fedora-based Linux distro for FireSim, do the following:
|
||||
FireMarshal comes with a Fedora-based workload that you can use right out of
|
||||
the box in ``workloads/fedora-base.json``. We begin by building the
|
||||
workload (filesystem and boot-binary):
|
||||
|
||||
::
|
||||
|
||||
cd firesim/sw/firesim-software
|
||||
./sw-manager.py -c fedora-disk.json build
|
||||
./marshal build workloads/fedora-base.json
|
||||
|
||||
Testing or customizing the target software using QEMU
|
||||
-----------------------------------------------------
|
||||
Before running this target software on FireSim, you may choose to boot the
|
||||
image in QEMU (a high-performance functional simulator). From here, you will
|
||||
have access to the internet and can install packages (e.g. by using ``dnf
|
||||
install foo``), download files, or perform any configuration tasks you'd like
|
||||
before booting in FireSim. To boot an image in QEMU, simply use the launch
|
||||
command:
|
||||
The first time you build a workload may take a long time (we need to download
|
||||
and decompress a pre-built fedora image), but subsequent builds of the same
|
||||
base will use cached results. Once the command completes, you should see two
|
||||
new files in ``images/``: ``fedora-base-bin`` and ``fedora-base.img``. These
|
||||
are the boot-binary (linux + boot loader) and root filesystem (respectively).
|
||||
We can now launch this workload in qemu:
|
||||
|
||||
::
|
||||
|
||||
./sw-manager.py -c fedora-disk.json launch
|
||||
./marshal launch workloads/fedora-base.json
|
||||
|
||||
You should now see linux booting and be presented with a login prompt. Sign in
|
||||
as 'root' with password 'firesim'. From here you can download files, use the
|
||||
package manager (e.g. 'dnf install'), and generally use the image as if it had
|
||||
booted on real hardware with an internet connection. Any changes you make here
|
||||
will be persistent between reboots. Once you are done exploring, simply
|
||||
shutdown the workload:
|
||||
|
||||
Booting Fedora on FireSim
|
||||
----------------------------
|
||||
::
|
||||
|
||||
In order to boot Fedora on FireSim, change your workload to
|
||||
``fedora-uniform.json`` in runtime_config.ini and boot as usual.
|
||||
$ poweroff
|
||||
|
||||
It is typically not a good idea to modify the \*-base workloads directly since
|
||||
many other workloads might inherit those changes. To make sure that we've
|
||||
cleaned out any changes, let's clean and rebuild the workload:
|
||||
|
||||
::
|
||||
|
||||
./marshal clean workloads/fedora-base.json
|
||||
./marshal build workloads/fedora-base.json
|
||||
|
||||
Note that this build took significantly less time than the first; FireMarshal
|
||||
caches intermediate build steps whenever possible. The final step is to run
|
||||
this workload on the real firesim RTL with full timing accuracy. For the basic
|
||||
fedora distribution, we will use the pre-made firesim config at
|
||||
``firesim/deploy/workloads/fedora-uniform.json``. Simply change the
|
||||
``workloadname`` option in ``firesim/deploy/config_runtime.ini`` to
|
||||
"fedora-uniform.json" and then follow the standard FireSim procedure for
|
||||
booting a workload (e.g. :ref:`single-node-sim` or :ref:`cluster-sim`).
|
||||
|
||||
.. attention:: For the standard distributions we provide pre-built firesim
|
||||
workloads. In general, FireMarshal can derive a FireSim workload from
|
||||
the FireMarshal configuration using the ``install`` command (see
|
||||
:ref:`firemarshal-commands`)
|
||||
|
|
|
|||
|
|
@ -32,9 +32,10 @@ this should really be named "jobs" -- we will fix this in a future release.
|
|||
|
||||
**ERRATA**: The following instructions assume the default buildroot-based linux
|
||||
distribution (br-disk). In order to customize Fedora, you should build the
|
||||
basic Fedora image (as described in :ref:`booting-fedora`) and modify the
|
||||
image directly (or in QEMU). Imporantly, Fedora currently does not support the
|
||||
"command" option for workloads.
|
||||
basic Fedora image (as described in :ref:`booting-fedora`) and modify the image
|
||||
directly (or use :ref:`FireMarshal <firemarshal>` to generate the
|
||||
workload). Imporantly, Fedora currently does not support the "command" option
|
||||
for workloads.
|
||||
|
||||
Uniform Workload JSON
|
||||
----------------------------
|
||||
|
|
@ -55,8 +56,8 @@ There is also a corresponding directory named after this workload/file:
|
|||
total 4
|
||||
drwxrwxr-x 2 centos centos 42 May 17 21:58 .
|
||||
drwxrwxr-x 13 centos centos 4096 May 18 17:14 ..
|
||||
lrwxrwxrwx 1 centos centos 41 May 17 21:58 br-disk-bin -> ../../../sw/firesim-software/images/br-disk-bin
|
||||
lrwxrwxrwx 1 centos centos 41 May 17 21:58 br-disk.img -> ../../../sw/firesim-software/images/br-disk.img
|
||||
lrwxrwxrwx 1 centos centos 41 May 17 21:58 br-base-bin -> ../../../sw/firesim-software/images/br-base-bin
|
||||
lrwxrwxrwx 1 centos centos 41 May 17 21:58 br-base.img -> ../../../sw/firesim-software/images/br-base.img
|
||||
|
||||
We will elaborate on this later.
|
||||
|
||||
|
|
@ -75,8 +76,8 @@ in this workload are expected to boot from. The manager will copy this binary
|
|||
for each of the nodes in the simulation (each gets its own copy). The ``common_bootbinary`` path is
|
||||
relative to the workload's directory, in this case
|
||||
``firesim/deploy/workloads/linux-uniform``. You'll notice in the above output
|
||||
from ``ls -la`` that this is actually just a symlink to ``br-disk-bin`` that
|
||||
is built by the FireSim Linux distro in ``firesim/sw/firesim-software``.
|
||||
from ``ls -la`` that this is actually just a symlink to ``br-base-bin`` that
|
||||
is built by the :ref:`FireMarshal <firemarshal>` tool.
|
||||
|
||||
Similarly, the ``common_rootfs`` field represents the disk image that the simulations
|
||||
in this workload are expected to boot from. The manager will copy this root
|
||||
|
|
@ -84,8 +85,8 @@ filesystem image for each of the nodes in the simulation (each gets its own copy
|
|||
The ``common_rootfs`` path is
|
||||
relative to the workload's directory, in this case
|
||||
``firesim/deploy/workloads/linux-uniform``. You'll notice in the above output
|
||||
from ``ls -la`` that this is actually just a symlink to ``br-disk.img`` that
|
||||
is built by the FireSim Linux distro in ``firesim/sw/firesim-software``.
|
||||
from ``ls -la`` that this is actually just a symlink to ``br-base.img`` that
|
||||
is built by the :ref:`FireMarshal <firemarshal>` tool.
|
||||
|
||||
The ``common_outputs`` field is a list of outputs that the manager will copy out of
|
||||
the root filesystem image AFTER a simulation completes. In this simple example,
|
||||
|
|
@ -131,7 +132,7 @@ AFTER the workload is built:
|
|||
total 15203216
|
||||
drwxrwxr-x 3 centos centos 4096 May 18 07:45 .
|
||||
drwxrwxr-x 13 centos centos 4096 May 18 17:14 ..
|
||||
lrwxrwxrwx 1 centos centos 41 May 17 21:58 bbl-vmlinux -> ../../../sw/firesim-software/images/br-disk-bin
|
||||
lrwxrwxrwx 1 centos centos 41 May 17 21:58 bbl-vmlinux -> ../linux-uniform/br-base-bin
|
||||
-rw-rw-r-- 1 centos centos 7 May 17 21:58 .gitignore
|
||||
-rw-r--r-- 1 centos centos 1946009600 May 18 07:45 idler-1.ext2
|
||||
-rw-r--r-- 1 centos centos 1946009600 May 18 07:45 idler-2.ext2
|
||||
|
|
@ -147,7 +148,7 @@ AFTER the workload is built:
|
|||
|
||||
First, let's identify some of these files:
|
||||
|
||||
- ``bbl-vmlinux``: Just like in the ``linux-uniform`` case, this workload just uses the default Linux binary generated in ``firesim-software``. Note that it's named differently here, but still symlinks to ``br-disk-bin`` in firesim-software.
|
||||
- ``bbl-vmlinux``: This workload just uses the default linux binary generated for the ``linux-uniform`` workload.
|
||||
- ``.gitignore``: This just ignores the generated rootfses, which we'll learn about below.
|
||||
- ``idler-[1-6].ext2``, ``pingee.ext2``, ``pinger.ext2``: These are rootfses that are generated from the json script above. We'll learn how to do this shortly.
|
||||
|
||||
|
|
@ -186,7 +187,7 @@ see in the ``ping-latency`` directory.
|
|||
::
|
||||
|
||||
[ from the workloads/ directory ]
|
||||
python gen-benchmark-rootfs.py -w ping-latency.json -r -b ../../sw/firesim-software/images/br-disk.img -s ping-latency/overlay
|
||||
python gen-benchmark-rootfs.py -w ping-latency.json -r -b ../../sw/firesim-software/images/br-base.img -s ping-latency/overlay
|
||||
|
||||
Notice that we tell this script where the json file lives, where the base rootfs image is, and where we expect to find files
|
||||
that we want to include in the generated disk images. This script will take care of the rest and we'll end up with
|
||||
|
|
|
|||
|
|
@ -0,0 +1,42 @@
|
|||
.. _gap-benchmark-suite:
|
||||
|
||||
GAP Benchmark Suite
|
||||
---------------------
|
||||
You can run the reference implementation of the GAP (Graph Algorithm Performance)
|
||||
Benchmark Suite. We provide scripts that cross-compile the graph kernels for RISCV.
|
||||
|
||||
For more information about the benchmark itself, please refer to the site:
|
||||
http://gap.cs.berkeley.edu/benchmark.html
|
||||
|
||||
Some notes:
|
||||
|
||||
- Only the Kron input graph is currently supported.
|
||||
- Benchmark uses ``graph500`` input graph size of 2^20 vertices by default. ``test`` input size has 2^10 vertices and can be used by specifying an argument into make: ``make gapbs input=test``
|
||||
- The reference input size with 2^27 verticies is not currently supported.
|
||||
|
||||
By default, the gapbs workload definition runs the benchmark multithreaded with number of threads equal to the number of cores. To change the number of threads, you need to edit ``firesim/deploy/workloads/runscripts/gapbs-scripts/gapbs.sh``. Additionally, the workload does not verify the output of the benchmark by default. To change this, add a ``--verify`` parameter to the json.
|
||||
|
||||
To Build Binaries and RootFSes:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd firesim/deploy/workloads/
|
||||
make gapbs
|
||||
|
||||
Run Resource Requirements:
|
||||
|
||||
.. include:: /../deploy/workloads/gapbs.ini
|
||||
:start-line: 3
|
||||
:end-line: 6
|
||||
:code: ini
|
||||
|
||||
|
||||
To Run:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
./run-workload.sh workloads/gapbs.ini --withlaunch
|
||||
|
||||
Simulation times are host and target dependent. For reference, on a
|
||||
four-core rocket-based SoC with a DDR3 + 1 MiB LLC model, with a 90
|
||||
MHz host clock, ``test`` and ``graph500`` input sizes finish in a few minutes.
|
||||
|
|
@ -35,7 +35,7 @@ Building Benchmark Binaries/Rootfses
|
|||
|
||||
We include scripts to automatically build all of the benchmark rootfs images
|
||||
that will be used below. To build them, make sure you have already run
|
||||
``./sw-manager.py -c br-disk.json build`` in ``firesim/sw/firesim-software``, then run:
|
||||
``./marshal build workloads/br-base.json`` in ``firesim/sw/firesim-software``, then run:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
|
|
@ -119,7 +119,8 @@ To Run:
|
|||
./run-simperf-test-scale.sh withlaunch
|
||||
|
||||
|
||||
Notes: Excludes supernode since it is still in beta and not merged on master.
|
||||
A similar benchmark is also provided for supernode mode, see ``run-simperf-test-scale-supernode.sh``.
|
||||
|
||||
|
||||
Figure 9: Simulation Rate vs. Link Latency
|
||||
---------------------------------------------
|
||||
|
|
@ -140,7 +141,7 @@ To Run:
|
|||
./run-simperf-test-latency.sh withlaunch
|
||||
|
||||
|
||||
Notes: Excludes supernode since it is still in beta and not merged on master.
|
||||
A similar benchmark for supernode mode will be provided soon. See https://github.com/firesim/firesim/issues/244
|
||||
|
||||
|
||||
Running all experiments at once
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@ EC2.
|
|||
|
||||
Some notes:
|
||||
|
||||
- Benchmarks use reference inputs. ``train`` or ``test`` inputs can be used by changing the Speckle invocation in the Makefile.
|
||||
- Benchmarks use reference inputs by default. ``train`` or ``test`` inputs can be used by specifying an argument in make: ``make spec-int{rate,speed} input={test,train,ref}``
|
||||
- You may need to increase the size of the RootFS in buildroot in ``firesim/sw/firesim-software/images``.
|
||||
- No support for fp{rate, speed} benchmarks yet.
|
||||
|
||||
|
|
@ -50,10 +50,7 @@ To Run:
|
|||
|
||||
.. code-block:: bash
|
||||
|
||||
firesim launchrunfarm -c workloads/spec17-intspeed.ini
|
||||
firesim infrasetup -c workloads/spec17-intspeed.ini
|
||||
firesim runworkload -c workloads/spec17-intspeed.ini
|
||||
firesim terminaterunfarm -c workloads/spec17-intspeed.ini
|
||||
./run-workload.sh workloads/spec17-intspeed.ini --withlaunch
|
||||
|
||||
On a single-core rocket-based SoC with a DDR3 + 256 KiB LLC model, with a 160
|
||||
MHz host clock, the longest benchmarks (xz, mcf) complete in about 1
|
||||
|
|
@ -86,10 +83,7 @@ To Run:
|
|||
|
||||
.. code-block:: bash
|
||||
|
||||
firesim launchrunfarm -c workloads/spec17-intrate.ini
|
||||
firesim infrasetup -c workloads/spec17-intrate.ini
|
||||
firesim runworkload -c workloads/spec17-intrate.ini
|
||||
firesim terminaterunfarm -c workloads/spec17-intrate.ini
|
||||
./run-workload.sh workloads/spec17-intrate.ini --withlaunch
|
||||
|
||||
|
||||
Simulation times are host and target dependent. For reference, on a
|
||||
|
|
|
|||
|
|
@ -1,8 +1,12 @@
|
|||
Workloads
|
||||
================
|
||||
|
||||
This section describes workload definitions in FireSim.
|
||||
.. attention::
|
||||
|
||||
FireSim is moving to a new workload-generation tool :ref:`firemarshal`.
|
||||
These instructions will be deprecated in future releases of FireSim.
|
||||
|
||||
This section describes workload definitions in FireSim.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
|
@ -13,3 +17,4 @@ This section describes workload definitions in FireSim.
|
|||
SPEC-2017
|
||||
Booting-Fedora
|
||||
ISCA-2018-Experiments
|
||||
GAP-Benchmark-Suite
|
||||
|
|
|
|||
|
|
@ -36,7 +36,7 @@ Check your EC2 Instance Limits
|
|||
|
||||
AWS limits access to particular instance types for new/infrequently used
|
||||
accounts to protect their infrastructure. You should make sure that your
|
||||
account has access to ``f1.2xlarge``, ``f1.16xlarge``,
|
||||
account has access to ``f1.2xlarge``, ``f1.4xlarge``, ``f1.16xlarge``,
|
||||
``m4.16xlarge``, and ``c4.4xlarge`` instances by looking at the "Limits" page
|
||||
in the EC2 panel, which you can access
|
||||
`here <https://console.aws.amazon.com/ec2/v2/home#Limits:>`__. The
|
||||
|
|
|
|||
|
|
@ -11,36 +11,22 @@ Since we will deploy the heavy lifting to separate ``c4.4xlarge`` and
|
|||
we will use a ``c4.4xlarge``,
|
||||
running the AWS FPGA Developer AMI (be sure to subscribe if you have not done so. See :ref:`ami-subscription`).
|
||||
|
||||
.. Head to the `EC2 Management
|
||||
.. Console <https://console.aws.amazon.com/ec2/v2/home>`__. In the top
|
||||
.. right corner, ensure that the correct region is selected.
|
||||
.. 1. From the main page of the EC2 Management Console, click
|
||||
``Launch Instance``. We use an on-demand instance here, so that your
|
||||
data is preserved when you stop/start the instance, and your data is
|
||||
not lost when pricing spikes on the spot market.
|
||||
.. 6. When prompted to select an AMI, search in the ``Community AMIs`` tab for
|
||||
"FPGA" and select the option that starts with ``FPGA Developer AMI - 1.4.0``.
|
||||
**DO NOT USE ANY OTHER VERSION.**
|
||||
Head to the `EC2 Management
|
||||
Console <https://console.aws.amazon.com/ec2/v2/home>`__. In the top
|
||||
right corner, ensure that the correct region is selected.
|
||||
|
||||
To launch a manager instance, follow these steps:
|
||||
|
||||
1. Head to the FPGA Developer AMI Page on AWS Marketplace:
|
||||
`https://aws.amazon.com/marketplace/pp/B06VVYBLZZ <https://aws.amazon.com/marketplace/pp/B06VVYBLZZ>`__
|
||||
2. Click the ``Continue to Subscribe`` button in the top-right.
|
||||
3. On the following page, select ``Continue to Configuration``.
|
||||
4. On the following page, you will be presented with several dropdown menus:
|
||||
|
||||
1. Do not change ``Fulfillment Option``.
|
||||
2. For ``Software Version``, select ``1.4.0 (May 08, 2018)``. You MUST use this version. **DO NOT USE ANY OTHER VERSION.**
|
||||
3. For ``Region``, select your desired region.
|
||||
4. Click ``Continue to Launch`` in the top-right.
|
||||
5. On the following page, in the ``Choose Action`` dropdown, select ``Launch
|
||||
through EC2``. Upon doing this, the options after this dropdown will
|
||||
disappear and you will be presented with a ``Launch`` button, which you
|
||||
should click.
|
||||
6. When prompted to choose an instance type, select the instance type of
|
||||
1. From the main page of the EC2 Management Console, click
|
||||
``Launch Instance``. We use an on-demand instance here, so that your
|
||||
data is preserved when you stop/start the instance, and your data is
|
||||
not lost when pricing spikes on the spot market.
|
||||
2. When prompted to select an AMI, search in the ``Community AMIs`` tab for
|
||||
"FPGA" and select the option that starts with ``FPGA Developer AMI - 1.5.0``.
|
||||
**DO NOT USE ANY OTHER VERSION.**
|
||||
3. When prompted to choose an instance type, select the instance type of
|
||||
your choosing. A good choice is a ``c4.4xlarge``.
|
||||
7. On the "Configure Instance Details" page:
|
||||
4. On the "Configure Instance Details" page:
|
||||
|
||||
1. First make sure that the ``firesim`` VPC is selected in the
|
||||
drop-down box next to "Network". Any subnet within the ``firesim``
|
||||
|
|
@ -58,16 +44,17 @@ To launch a manager instance, follow these steps:
|
|||
|
||||
This will pre-install all of the dependencies needed to run FireSim on your instance.
|
||||
|
||||
8. On the next page ("Add Storage"), increase the size of the root EBS
|
||||
5. On the next page ("Add Storage"), increase the size of the root EBS
|
||||
volume to ~300GB. The default of 150GB can quickly become tight as
|
||||
you accumulate large Vivado reports/outputs, large waveforms, XSim outputs,
|
||||
and large root filesystems for simulations. You can get rid of the
|
||||
small (5GB) secondary volume that is added by default.
|
||||
9. You can skip the "Add Tags" page, unless you want tags.
|
||||
10. On the "Configure Security Group" page, select the ``firesim``
|
||||
security group that was automatically created for you earlier.
|
||||
11. On the review page, click the button to launch your instance.
|
||||
**Make sure you select the** ``firesim`` **key pair that we setup earlier.**
|
||||
6. You can skip the "Add Tags" page, unless you want tags.
|
||||
7. On the "Configure Security Group" page, select the ``firesim``
|
||||
security group that was automatically created for you earlier.
|
||||
8. On the review page, click the button to launch your instance.
|
||||
|
||||
Make sure you select the ``firesim`` key pair that we setup earlier.
|
||||
|
||||
Access your instance
|
||||
~~~~~~~~~~~~~~~~~~~~
|
||||
|
|
|
|||
|
|
@ -37,7 +37,7 @@ this like so:
|
|||
::
|
||||
|
||||
cd firesim/sw/firesim-software
|
||||
./sw-manager.py -c br-disk.json build
|
||||
./marshal -v build workloads/br-base.json
|
||||
|
||||
This process will take about 10 to 15 minutes on a ``c4.4xlarge`` instance.
|
||||
Once this is completed, you'll have the following files:
|
||||
|
|
@ -127,6 +127,7 @@ You should expect output like the following:
|
|||
|
||||
Waiting for instance boots: f1.16xlarges
|
||||
i-09e5491cce4d5f92d booted!
|
||||
Waiting for instance boots: f1.4xlarges
|
||||
Waiting for instance boots: m4.16xlarges
|
||||
Waiting for instance boots: f1.2xlarges
|
||||
The full log of this run is:
|
||||
|
|
@ -181,8 +182,9 @@ For a complete run, you should expect output like the following:
|
|||
[172.30.2.178] Copying FPGA simulation infrastructure for slot: 6.
|
||||
[172.30.2.178] Copying FPGA simulation infrastructure for slot: 7.
|
||||
[172.30.2.178] Installing AWS FPGA SDK on remote nodes.
|
||||
[172.30.2.178] Unloading EDMA Driver Kernel Module.
|
||||
[172.30.2.178] Copying AWS FPGA EDMA driver to remote node.
|
||||
[172.30.2.178] Unloading XDMA/EDMA/XOCL Driver Kernel Module.
|
||||
[172.30.2.178] Copying AWS FPGA XDMA driver to remote node.
|
||||
[172.30.2.178] Loading XDMA Driver Kernel Module.
|
||||
[172.30.2.178] Clearing FPGA Slot 0.
|
||||
[172.30.2.178] Clearing FPGA Slot 1.
|
||||
[172.30.2.178] Clearing FPGA Slot 2.
|
||||
|
|
@ -199,7 +201,8 @@ For a complete run, you should expect output like the following:
|
|||
[172.30.2.178] Flashing FPGA Slot: 5 with agfi: agfi-09e85ffabe3543903.
|
||||
[172.30.2.178] Flashing FPGA Slot: 6 with agfi: agfi-09e85ffabe3543903.
|
||||
[172.30.2.178] Flashing FPGA Slot: 7 with agfi: agfi-09e85ffabe3543903.
|
||||
[172.30.2.178] Loading EDMA Driver Kernel Module.
|
||||
[172.30.2.178] Unloading XDMA/EDMA/XOCL Driver Kernel Module.
|
||||
[172.30.2.178] Loading XDMA Driver Kernel Module.
|
||||
[172.30.2.178] Copying switch simulation infrastructure for switch slot: 0.
|
||||
The full log of this run is:
|
||||
/home/centos/firesim-new/deploy/logs/2018-05-19--06-07-33-infrasetup-2Z7EBCBIF2TSI66Q.log
|
||||
|
|
@ -514,6 +517,8 @@ Which should present you with the following:
|
|||
IMPORTANT!: This will terminate the following instances:
|
||||
f1.16xlarges
|
||||
['i-09e5491cce4d5f92d']
|
||||
f1.4xlarges
|
||||
[]
|
||||
m4.16xlarges
|
||||
[]
|
||||
f1.2xlarges
|
||||
|
|
|
|||
|
|
@ -22,7 +22,7 @@ distribution. You can do this like so:
|
|||
::
|
||||
|
||||
cd firesim/sw/firesim-software
|
||||
./sw-manager.py -c br-disk.json build
|
||||
./marshal -v build workloads/br-base.json
|
||||
|
||||
This process will take about 10 to 15 minutes on a ``c4.4xlarge`` instance.
|
||||
Once this is completed, you'll have the following files:
|
||||
|
|
@ -55,7 +55,7 @@ We'll need to modify a couple of these lines.
|
|||
First, let's tell the manager to use the correct numbers and types of instances.
|
||||
You'll notice that in the ``[runfarm]`` section, the manager is configured to
|
||||
launch a Run Farm named ``mainrunfarm``, consisting of one ``f1.16xlarge`` and
|
||||
no ``m4.16xlarge``\ s or ``f1.2xlarge``\ s. The tag specified here allows the
|
||||
no ``m4.16xlarge``\ s, ``f1.4xlarge``\ s, or ``f1.2xlarge``\ s. The tag specified here allows the
|
||||
manager to differentiate amongst many parallel run farms (each running
|
||||
a workload) that you may be operating -- but more on that later.
|
||||
|
||||
|
|
@ -68,6 +68,7 @@ Since we only want to simulate a single node, let's switch to using one
|
|||
# per aws restrictions, this tag cannot be longer than 255 chars
|
||||
runfarmtag=mainrunfarm
|
||||
f1_16xlarges=0
|
||||
f1_4xlarges=0
|
||||
m4_16xlarges=0
|
||||
f1_2xlarges=1
|
||||
|
||||
|
|
@ -122,6 +123,7 @@ As a final sanity check, your ``config_runtime.ini`` file should now look like t
|
|||
runfarmtag=mainrunfarm
|
||||
|
||||
f1_16xlarges=0
|
||||
f1_4xlarges=1
|
||||
m4_16xlarges=0
|
||||
f1_2xlarges=1
|
||||
|
||||
|
|
@ -181,6 +183,7 @@ You should expect output like the following:
|
|||
Running: launchrunfarm
|
||||
|
||||
Waiting for instance boots: f1.16xlarges
|
||||
Waiting for instance boots: f1.4xlarges
|
||||
Waiting for instance boots: m4.16xlarges
|
||||
Waiting for instance boots: f1.2xlarges
|
||||
i-0d6c29ac507139163 booted!
|
||||
|
|
@ -227,11 +230,13 @@ For a complete run, you should expect output like the following:
|
|||
[172.30.2.174] Executing task 'infrasetup_node_wrapper'
|
||||
[172.30.2.174] Copying FPGA simulation infrastructure for slot: 0.
|
||||
[172.30.2.174] Installing AWS FPGA SDK on remote nodes.
|
||||
[172.30.2.174] Unloading EDMA Driver Kernel Module.
|
||||
[172.30.2.174] Copying AWS FPGA EDMA driver to remote node.
|
||||
[172.30.2.174] Unloading XDMA/EDMA/XOCL Driver Kernel Module.
|
||||
[172.30.2.174] Copying AWS FPGA XDMA driver to remote node.
|
||||
[172.30.2.174] Loading XDMA Driver Kernel Module.
|
||||
[172.30.2.174] Clearing FPGA Slot 0.
|
||||
[172.30.2.174] Flashing FPGA Slot: 0 with agfi: agfi-0eaa90f6bb893c0f7.
|
||||
[172.30.2.174] Loading EDMA Driver Kernel Module.
|
||||
[172.30.2.174] Unloading XDMA/EDMA/XOCL Driver Kernel Module.
|
||||
[172.30.2.174] Loading XDMA Driver Kernel Module.
|
||||
The full log of this run is:
|
||||
/home/centos/firesim-new/deploy/logs/2018-05-19--00-32-02-infrasetup-9DJJCX29PF4GAIVL.log
|
||||
|
||||
|
|
@ -465,6 +470,8 @@ Which should present you with the following:
|
|||
IMPORTANT!: This will terminate the following instances:
|
||||
f1.16xlarges
|
||||
[]
|
||||
f1.4xlarges
|
||||
[]
|
||||
m4.16xlarges
|
||||
[]
|
||||
f1.2xlarges
|
||||
|
|
|
|||
|
|
@ -1,3 +1,5 @@
|
|||
.. _running_simulations:
|
||||
|
||||
Running FireSim Simulations
|
||||
================================
|
||||
|
||||
|
|
|
|||
20
docs/conf.py
20
docs/conf.py
|
|
@ -6,6 +6,9 @@
|
|||
# full list see the documentation:
|
||||
# http://www.sphinx-doc.org/en/master/config
|
||||
|
||||
import shutil
|
||||
import os
|
||||
|
||||
# -- Path setup --------------------------------------------------------------
|
||||
|
||||
# If extensions (or modules to document with autodoc) are in another directory,
|
||||
|
|
@ -160,3 +163,20 @@ texinfo_documents = [
|
|||
author, 'FireSim', 'One line description of project.',
|
||||
'Miscellaneous'),
|
||||
]
|
||||
|
||||
# -- handle re-directs for pages that move
|
||||
# taken from https://tech.signavio.com/2017/managing-sphinx-redirects
|
||||
|
||||
redirect_files = [ ]
|
||||
|
||||
def copy_legacy_redirects(app, docname): # Sphinx expects two arguments
|
||||
if app.builder.name == 'html':
|
||||
for html_src_path in redirect_files:
|
||||
target_path = app.outdir + '/' + html_src_path
|
||||
src_path = app.srcdir + '/' + html_src_path
|
||||
|
||||
if os.path.isfile(src_path):
|
||||
shutil.copyfile(src_path, target_path)
|
||||
|
||||
def setup(app):
|
||||
app.connect('build-finished', copy_legacy_redirects)
|
||||
|
|
|
|||
|
|
@ -24,6 +24,7 @@ New to FireSim? Jump to the :ref:`firesim-basics` page for more info.
|
|||
|
||||
Advanced-Usage/Manager/index
|
||||
Advanced-Usage/Workloads/index
|
||||
Advanced-Usage/FireMarshal/index
|
||||
Advanced-Usage/Generating-Different-Targets.rst
|
||||
Advanced-Usage/Debugging/index
|
||||
Developing-New-Devices/index
|
||||
|
|
|
|||
|
|
@ -1 +1 @@
|
|||
Subproject commit bca30e63c91b30e6db9b253077da805ca69513e7
|
||||
Subproject commit 1b6c0d420afd5327850cb5d374a9d666c68ad1f8
|
||||
|
|
@ -0,0 +1,16 @@
|
|||
#!/usr/bin/env bash
|
||||
|
||||
set -e
|
||||
set -o pipefail
|
||||
|
||||
# build setup
|
||||
./build-setup.sh fast
|
||||
source sourceme-f1-manager.sh
|
||||
|
||||
# run through elaboration flow to get chisel/sbt all setup
|
||||
cd sim
|
||||
make f1
|
||||
|
||||
# build target software
|
||||
cd ../sw/firesim-software
|
||||
./marshal -v build workloads/br-base.json
|
||||
|
|
@ -10,6 +10,8 @@ sudo yum install -y expat-devel libusb1-devel ncurses-devel cmake "perl(ExtUtils
|
|||
sudo yum install -y python34 patch diffstat texi2html texinfo subversion chrpath git wget
|
||||
# deps for qemu
|
||||
sudo yum install -y gtk3-devel
|
||||
# deps for firesim-software (note that rsync is installed but too old)
|
||||
sudo yum install -y python34-pip python34-devel rsync
|
||||
# install DTC. it's not available in repos in FPGA AMI
|
||||
DTCversion=dtc-1.4.4
|
||||
wget https://git.kernel.org/pub/scm/utils/dtc/dtc.git/snapshot/$DTCversion.tar.gz
|
||||
|
|
@ -41,14 +43,17 @@ sudo yum -y install bash-completion
|
|||
sudo yum -y install graphviz python-devel
|
||||
|
||||
# these need to match what's in deploy/requirements.txt
|
||||
sudo pip install fabric==1.14.0
|
||||
sudo pip install boto3==1.6.2
|
||||
sudo pip install colorama==0.3.7
|
||||
sudo pip install argcomplete==1.9.3
|
||||
sudo pip install graphviz==0.8.3
|
||||
sudo pip2 install fabric==1.14.0
|
||||
sudo pip2 install boto3==1.6.2
|
||||
sudo pip2 install colorama==0.3.7
|
||||
sudo pip2 install argcomplete==1.9.3
|
||||
sudo pip2 install graphviz==0.8.3
|
||||
# for some of our workload plotting scripts
|
||||
sudo pip install matplotlib==2.2.2
|
||||
sudo pip install pandas==0.22.0
|
||||
sudo pip2 install --upgrade --ignore-installed pyparsing
|
||||
sudo pip2 install matplotlib==2.2.2
|
||||
sudo pip2 install pandas==0.22.0
|
||||
# this is explicitly installed to downgrade it to a version without deprec warnings
|
||||
sudo pip2 install cryptography==2.2.2
|
||||
|
||||
sudo activate-global-python-argcomplete
|
||||
|
||||
|
|
|
|||
|
|
@ -34,61 +34,76 @@ export DESIGN=FireSimNoNIC
|
|||
export TARGET_CONFIG=FireSimRocketChipConfig
|
||||
export PLATFORM_CONFIG=FireSimConfig
|
||||
export SIM_ARGS=+verbose
|
||||
export TIME="%C %E real, %U user, %S sys"
|
||||
|
||||
## Verilator
|
||||
cd $firesim_root/target-design/firechip/verisim
|
||||
sim=simulator-example-DefaultExampleConfig
|
||||
make -j$MAKE_THREADS
|
||||
make -j$MAKE_THREADS debug
|
||||
for optlevel in 0 1 2
|
||||
do
|
||||
|
||||
/usr/bin/time -a -o nowaves.log ./$sim $SIM_ARGS $test_path &> nowaves.log
|
||||
/usr/bin/time -a -o waves.log ./$sim-debug $SIM_ARGS -vtest.vcd $test_path &> waves.log
|
||||
echo -e "\nVerilator TARGET-level Simulation, -O${optlevel}\n" >> $REPORT_FILE
|
||||
## Verilator
|
||||
cd $firesim_root/target-design/firechip/verisim
|
||||
sim=simulator-example-DefaultExampleConfig
|
||||
|
||||
echo -e "\nTarget-level Verilator\n" >> $REPORT_FILE
|
||||
tail nowaves.log >> $REPORT_FILE
|
||||
echo -e "\nTarget-level Verilator -- Waves Enabled\n" >> $REPORT_FILE
|
||||
tail waves.log >> $REPORT_FILE
|
||||
# Hack...
|
||||
sed -i "s/-O[0-3]/-O${optlevel}/" Makefile
|
||||
make clean
|
||||
/usr/bin/time -a -o $REPORT_FILE make
|
||||
/usr/bin/time -a -o $REPORT_FILE make debug
|
||||
|
||||
## VCS
|
||||
cd $firesim_root/target-design/firechip/vsim/
|
||||
sim=simv-example-DefaultExampleConfig
|
||||
make -j$MAKE_THREADS
|
||||
make -j$MAKE_THREADS debug
|
||||
|
||||
./$sim $SIM_ARGS $test_path &> nowaves.log
|
||||
./$sim-debug $SIM_ARGS $test_path &> waves.log
|
||||
echo -e "\nNo Waves\n" >> $REPORT_FILE
|
||||
/usr/bin/time -a -o $REPORT_FILE ./$sim $SIM_ARGS $test_path &> nowaves.log
|
||||
tail nowaves.log >> $REPORT_FILE
|
||||
/usr/bin/time -a -o $REPORT_FILE ./$sim-debug $SIM_ARGS -vtest.vcd $test_path &> waves.log
|
||||
echo -e "\nWaves Enabled\n" >> $REPORT_FILE
|
||||
tail waves.log >> $REPORT_FILE
|
||||
done
|
||||
|
||||
echo -e "\nTarget-level VCS\n" >> $REPORT_FILE
|
||||
cd $firesim_root/target-design/firechip/vsim/
|
||||
sim=simv-example-DefaultExampleConfig
|
||||
/usr/bin/time -a -o $REPORT_FILE make -j$MAKE_THREADS
|
||||
/usr/bin/time -a -o $REPORT_FILE make -j$MAKE_THREADS debug
|
||||
|
||||
echo -e "\nNo Waves\n" >> $REPORT_FILE
|
||||
/usr/bin/time -a -o $REPORT_FILE ./$sim $SIM_ARGS $test_path &> nowaves.log
|
||||
tail nowaves.log >> $REPORT_FILE
|
||||
echo -e "\nTarget-level VCS -- Waves Enabled\n" >> $REPORT_FILE
|
||||
echo -e "\nWaves Enabled\n" >> $REPORT_FILE
|
||||
/usr/bin/time -a -o $REPORT_FILE ./$sim-debug $SIM_ARGS $test_path &> waves.log
|
||||
tail waves.log >> $REPORT_FILE
|
||||
|
||||
#################################################################################
|
||||
################################################################################
|
||||
## MIDAS level
|
||||
################################################################################
|
||||
ml_output_dir=$firesim_root/sim/output/f1/$DESIGN-$TARGET_CONFIG-$PLATFORM_CONFIG
|
||||
test_symlink=$ml_output_dir/$TEST
|
||||
|
||||
cd $firesim_root/sim
|
||||
make -j$MAKE_THREADS verilator
|
||||
make -j$MAKE_THREADS verilator-debug
|
||||
make -j$MAKE_THREADS vcs
|
||||
make -j$MAKE_THREADS vcs-debug
|
||||
mkdir -p $ml_output_dir
|
||||
for optlevel in 0 1 2
|
||||
do
|
||||
echo -e "\nMIDAS-level Simulation, -O${optlevel}\n" >> $REPORT_FILE
|
||||
cd $firesim_root/sim
|
||||
make clean
|
||||
make -j$MAKE_THREADS
|
||||
/usr/bin/time -a -o $REPORT_FILE make -j$MAKE_THREADS VERILATOR_CXXOPTS=-O${optlevel} verilator
|
||||
/usr/bin/time -a -o $REPORT_FILE make -j$MAKE_THREADS VERILATOR_CXXOPTS=-O${optlevel} verilator-debug
|
||||
/usr/bin/time -a -o $REPORT_FILE make -j$MAKE_THREADS VCS_CXXOPTS=-O${optlevel} vcs
|
||||
/usr/bin/time -a -o $REPORT_FILE make -j$MAKE_THREADS VCS_CXXOPTS=-O${optlevel} vcs-debug
|
||||
mkdir -p $ml_output_dir
|
||||
|
||||
# Symlink it twice so we have unique targets for vcs and verilator
|
||||
ln -sf $test_path $ml_output_dir/$TEST
|
||||
ln -sf $test_path $ml_output_dir/$TEST-vcs
|
||||
# Symlink it twice so we have unique targets for vcs and verilator
|
||||
ln -sf $test_path $ml_output_dir/$TEST
|
||||
ln -sf $test_path $ml_output_dir/$TEST-vcs
|
||||
|
||||
echo -e "\nMIDAS-level Waves Off\n" >> $REPORT_FILE
|
||||
make EMUL=vcs ${test_symlink}-vcs.out
|
||||
make ${test_symlink}.out
|
||||
grep -Eo "simulation speed = .*" $ml_output_dir/*out >> $REPORT_FILE
|
||||
echo -e "\nWaves Off, -O${optlevel}\n" >> $REPORT_FILE
|
||||
make EMUL=vcs ${test_symlink}-vcs.out
|
||||
make ${test_symlink}.out
|
||||
grep -Eo "simulation speed = .*" $ml_output_dir/*out >> $REPORT_FILE
|
||||
|
||||
echo -e "\nMIDAS-level Waves On\n" >> $REPORT_FILE
|
||||
make EMUL=vcs ${test_symlink}-vcs.vpd
|
||||
make ${test_symlink}.vpd
|
||||
grep -Eo "simulation speed = .*" $ml_output_dir/*out >> $REPORT_FILE
|
||||
echo -e "\nWaves On, -O${optlevel}\n" >> $REPORT_FILE
|
||||
make EMUL=vcs ${test_symlink}-vcs.vpd
|
||||
make ${test_symlink}.vpd
|
||||
grep -Eo "simulation speed = .*" $ml_output_dir/*out >> $REPORT_FILE
|
||||
done
|
||||
|
||||
################################################################################
|
||||
# FPGA level
|
||||
|
|
|
|||
|
|
@ -0,0 +1,12 @@
|
|||
#!/bin/bash
|
||||
# This script provides a quick and dirty means to capture the state of the
|
||||
# firesim repo and its submodules, when launching a build so that it can be
|
||||
# recreated manually later.
|
||||
git --no-pager log -n 1
|
||||
|
||||
if [[ -n $(git status -s) ]]; then
|
||||
echo -e "\nRepo is dirty. Diff of tracked files follows.\n"
|
||||
git --no-pager diff --submodule=diff
|
||||
else
|
||||
echo -e "\nRepo is clean"
|
||||
fi
|
||||
|
|
@ -6,3 +6,4 @@ tags
|
|||
*.timestamp
|
||||
AsyncResetReg.v
|
||||
firrtl_black_box_resource_files.f
|
||||
lib/firrtl.jar
|
||||
|
|
|
|||
32
sim/Makefile
32
sim/Makefile
|
|
@ -18,22 +18,28 @@ TARGET_PROJECT_MAKEFRAG ?= src/main/makefrag/$(TARGET_PROJECT)/Makefrag
|
|||
default: compile
|
||||
|
||||
SBT ?= sbt
|
||||
SBT_FLAGS ?= -J-Xmx16G -J-Xss8M -J-XX:MaxPermSize=256M -J-XX:MaxMetaspaceSize=512M -J-XX:ReservedCodeCacheSize=1G ++2.12.4
|
||||
JVM_MEMORY ?= 16G
|
||||
SBT_FLAGS ?= -J-Xmx$(JVM_MEMORY) ++2.12.4
|
||||
|
||||
sbt:
|
||||
$(SBT) $(SBT_FLAGS) shell
|
||||
test:
|
||||
$(SBT) $(SBT_FLAGS) test
|
||||
|
||||
########################
|
||||
# Timestamp & Patching #
|
||||
########################
|
||||
firesim_base_dir := $(abspath .)
|
||||
timestamps = $(addprefix $(firesim_base_dir)/, $(addsuffix .timestamp, firrtl))
|
||||
|
||||
$(firesim_base_dir)/firrtl.timestamp: $(shell find $(firesim_base_dir)/firrtl/$(src_path) -name "*.scala")
|
||||
cd $(firesim_base_dir)/firrtl && $(SBT) $(SBT_FLAGS) publishLocal
|
||||
touch $@
|
||||
# Manage the FIRRTL dependency manually
|
||||
FIRRTL_SUBMODULE_DIR ?= $(firesim_base_dir)/firrtl
|
||||
FIRRTL_JAR ?= $(FIRRTL_SUBMODULE_DIR)/utils/bin/firrtl.jar
|
||||
$(FIRRTL_JAR): $(shell find $(FIRRTL_SUBMODULE_DIR)/src/main/scala -iname "*.scala")
|
||||
$(MAKE) -C $(FIRRTL_SUBMODULE_DIR) SBT="$(SBT) $(SBT_FLAGS)" root_dir=$(FIRRTL_SUBMODULE_DIR) build-scala
|
||||
touch $(FIRRTL_JAR)
|
||||
mkdir -p $(firesim_base_dir)/lib
|
||||
cp -p $(FIRRTL_JAR) $(firesim_base_dir)/lib/
|
||||
|
||||
firrtl: $(FIRRTL_JAR)
|
||||
.PHONY: firrtl
|
||||
|
||||
# Phony targets for launching the sbt shell and running scalatests
|
||||
sbt: $(FIRRTL_JAR)
|
||||
$(SBT) $(SBT_FLAGS) shell
|
||||
test: $(FIRRTL_JAR)
|
||||
$(SBT) $(SBT_FLAGS) test
|
||||
|
||||
PLATFORM := f1
|
||||
|
||||
|
|
|
|||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue