replica
/
firesim
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

more workload stuff

This commit is contained in:
Sagar Karandikar 2018-05-20 23:26:09 +00:00
parent 4aaac0bf53
commit 7ed808d001
1 changed files with 82 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

88
docs/Advanced-Usage/Workloads/Defining-Custom-Workloads.rst
View File

@ -108,19 +108,93 @@ be fixed in a future release.
Non-uniform Workload JSON (explicit job per simulated node) Non-uniform Workload JSON (explicit job per simulated node)
--------------------------------------------------------------- ---------------------------------------------------------------
Now, we'll look at the ``ping-latency`` workload, which explicitly defines a
job per simulated node.
.. include:: /../deploy/workloads/ping-latency.json .. include:: /../deploy/workloads/ping-latency.json
:code: json :code: json
Additionally, let's take a look at the state of the ``ping-latency`` directory
AFTER the workload is built:
::
centos@ip-172-30-2-111.us-west-2.compute.internal:~/firesim-new/deploy/workloads/ping-latency$ ls -la
total 15203216
drwxrwxr-x 3 centos centos 4096 May 18 07:45 .
drwxrwxr-x 13 centos centos 4096 May 18 17:14 ..
lrwxrwxrwx 1 centos centos 41 May 17 21:58 bbl-vmlinux -> ../../../sw/firesim-software/bbl-vmlinux0
-rw-rw-r-- 1 centos centos 7 May 17 21:58 .gitignore
-rw-r--r-- 1 centos centos 1946009600 May 18 07:45 idler-1.ext2
-rw-r--r-- 1 centos centos 1946009600 May 18 07:45 idler-2.ext2
-rw-r--r-- 1 centos centos 1946009600 May 18 07:45 idler-3.ext2
-rw-r--r-- 1 centos centos 1946009600 May 18 07:45 idler-4.ext2
-rw-r--r-- 1 centos centos 1946009600 May 18 07:45 idler-5.ext2
-rw-r--r-- 1 centos centos 1946009600 May 18 07:46 idler-6.ext2
drwxrwxr-x 3 centos centos 16 May 17 21:58 overlay
-rw-r--r-- 1 centos centos 1946009600 May 18 07:44 pingee.ext2
-rw-r--r-- 1 centos centos 1946009600 May 18 07:44 pinger.ext2
-rw-rw-r-- 1 centos centos 2236 May 17 21:58 ping-latency-graph.py
First, let's identify some of these files:
- ``bbl-vmlinux``: Just like in the ``linux-uniform`` case, this workload just uses the default Linux binary generated in ``firesim-software``
- ``.gitignore``: This just ignores the generated rootfses, which we'll learn about below.
- ``idler-[1-6].ext2``, ``pingee.ext2``, ``pinger.ext2``: These are rootfses that are generated from the json script above. We'll learn how to do this shortly.
Additionally, let's look at the ``overlay`` subdirectory:
::
centos@ip-172-30-2-111.us-west-2.compute.internal:~/firesim-new/deploy/workloads/ping-latency/overlay$ ls -la */*
-rwxrwxr-x 1 centos centos 249 May 17 21:58 bin/pinglatency.sh
This is a file that's actually committed to the repo, that runs the benchmark we want to
run on one of our simulated systems. We'll see how this is used soon.
Now, let's take a look at how we got here. First, let's review some of the new
fields present in this JSON file:
- ``common_files``: This is an array of files that will be included in ALL of the job rootfses when they're built. This is relative to a path that we'll pass to the script that generates rootfses.
- ``workloads``: This time, you'll notice that we have this array, which is populated by objects that represent individual jobs. Each job has some additional fields:
- ``name``: In this case, jobs are each assigned a name manually. These names MUST BE UNIQUE within a particular workload.
- ``files``: Just like ``common_files``, but specific to this job.
- ``command``: This is the command that will be run automatically immediately when the simulation running this job boots up. This is usually the command that starts the workload we want.
- ``simulation_outputs``: Just like ``common_simulation_outputs``, but specific to this job.
- ``outputs``: Just like ``common_outputs``, but specific to this job.
In this example, we specify one node that boots up and runs the
``pinglatency.sh`` benchmark, then powers off cleanly and 7 nodes that just
idle waiting to be pinged.
Given this JSON description, our existing ``pinglatency.sh`` script in the
overlay directory, and the base rootfses generated in ``firesim-software``,
the following command will automatically generate all of the rootfses that you
see in the ``ping-latency`` directory.
::
[ from the workloads/ directory ]
python gen-benchmark-rootfs.py -w ping-latency.json -r -b ../../sw/firesim-software/rootfs0.ext2 -s ping-latency/overlay
Notice that we tell this script where the json file lives, where the base rootfs image is, and where we expect to find files
that we want to include in the generated disk images. This script will take care of the rest and we'll end up with
``idler-[1-6].ext2``, ``pingee.ext2``, and ``pinger.ext2``!
You'll notice a Makefile in the ``workloads/`` directory -- it contains many
similar commands for all of the workloads included with FireSim.
Once you generate the rootfses for this workload, you can run it with the manager
by setting ``workload=ping-latency.json`` in ``config_runtime.ini``. The manager
will automatically look for the generated rootfses (based on workload and job names
that it reads from the json) and distribute work appropriately.
Just like in the uniform case, it will copy back the results that we specify
in the json file. We'll end up with a directory in ``firesim/deploy/results-workload/``
named after the workload name, with a subdirectory named after each job in the workload,
which will contain the output files we want.
Although each simulation in such a workload boots the same job.