* driver_tar and xclbin keys can now be a URI
* non URI paths are also supported, with relative paths being relative to the deploy folder
* adds documentation about URI support
* URI are tried 4 times with a 1 second delay after failure
* run-basic-linux-poweroff CI job is now serialized behind run-parallel-verilator-metasims to improve CI reliability
Co-authored-by: Filip Stamenkovic <92741622+filipstamenkovic-sifive@users.noreply.github.com>
Co-authored-by: Abraham Gonzalez <abe.j.gonza@gmail.com>
Co-authored-by: Tim Snyder <timothy.snyder@sifive.com>
use fsspec to enable xclbin's to be one of any URI protocol
supported by the library or an installed add-on
Co-authored-by: Tim Snyder <snyder.tim@gmail.com>
Co-authored-by: Filip Stamenkovic <92741622+filipstamenkovic-sifive@users.noreply.github.com>
Co-authored-by: Abraham Gonzalez <abe.j.gonza@gmail.com>
Addition of config_build_recipes.yaml use by RuntimeConfig in #1076 broke
the pytests for config checking but somehow, it passed in the PR check.
Co-authored-by: Tim Snyder <snyder.tim@gmail.com>
* switch to using pytest-mock mocker fixture
* create a task_mocker that understands how to mock firesim.TASKS correctly
* write the remaining tests and fix the ones I didn't have correct in initial PR
Abe was curious whether boto would error if you give it a bogus instance
type when launching an instance. I added a test that shows what moto will
do. The boto test might be a good example for how to un-munge the environment
for actually using AWS in unit tests that don't mock it out.
When patching for testing, I was concerned about where to do the patching
and making the symbol table lookups throw seemed like a sensible thing. However,
now that I'm done writing the tests, this seems like it might be overkill
and that's why I'm leaving it in a separate commit in case I want to revert it.
* make it a decorator because that's what it really is
* unify BUILD and RUN tasks by introspecting the config class needed at registration time
* unify special cases by allowing tasks to not take a config class and tweaking terminaterunfarm
so that it only takes the RuntimeConfig (stuffed the args into it like we do with BuildConfigFile)
There are some tests that depend on setup done earlier in the GH workflow (i.e. managerinit being run)
Others that are unclear as to whether they are supposed to fail before buildafi has been run.
Haven't written most of the RUN_TASK tests yet.
* don't use globals() to dispatch the tasks, create a module-level dict for that purpose
* pytest toplevel conftest.py will drop a firesim.py -> firesim symlink for test importing
* construct_argument_parser() should return the parser without actually
parsing so that the testharness can use it with arglists
* mv deploy/tests/test_amis{_snake,}.json
_snake is related to the janky way I manually created MOTO_AMIS_PATH the first time
I'm replacing that with a customized version of the moto/scripts/get_amis.py script
* add scripts/update_test_amis.py as copied from moto
just so that the diff is inspectable later...
* Customize scripts/update_test_amis.py for FireSim
And use it to update deploy/tests/test_amis.json
And add a test to check whether test_amis.json needs to be updated
And make all of the tests that will fail confusingly depend on that test
* add comment about running script when you update the AMI
Co-authored-by: Tim Snyder <snyder.tim@gmail.com>
* make default timeout 0 to match legacy behavior when new
key is not present in runtime.ini
* Add tests requires
* moving scripts/aws-setup.py into a loadable module
* adding sure to machine-launch-script.sh for nicer assertion style
* Generalize awstools.get_instances_by_tag_type to not assume it is
being called with 'fsimcluster' value, uses tags dict. Created
awstools.get_run_instances_by_tag_type that uses old symantics and calls
the general function similar to launch_run_instances()
* Add `additive` parameter to `awstools.launch_instances()` that controls
whether `count` is the number to be launched in that call or a total number
of instances that should be reached including ones already launched that
match the `type` and `tags`
* First pass at porting to python3
* Fix import errors | Setup user argcomplete
* Update awstools CLI with user data file | Bump CI to use it
* Wait until launch is complete
* Add userdata as string | Use sudo for machine-launch-script
* Remove execute permissions on machine-launch-script
* Better match on machine-launch-script complete
* Revert python-devel removal
* Use python3 for pytests
* Update more python3 items
* Remove extra shebang
* Port docs to python3 and add to CI
* Add ISCA experiments to CI build check
* Use yum not apt-get
* Add make to doc section
* Bump multilate-loadgen for sysroot fix
* For BW test build don't use shebang
* Fix docs Makefile options
* Fix more doc warnings
* Add first set of regression tests
* Fix raw_input
* Regression bump | Run workload fix
* Add functools to topology
* Fix linux poweroff test (nic still has issues)
* Update regression scripts
* Ignore machine-launch-script.sh in regression area
* Fix map python3 issues
* Get rid of shebangs
* Fix more regressions
* Print machine-launch.log on fail | More clarification on user_data
* Transfer to CI some shorter regressions
* Add a manual approval to fpga based tests
* Fix indentation in config.yml
* Fix test symlink
* Use spot for CI manager instance | Try to use python3 for aws CI container | Version all pip packages
* Make run-ini-api-tests an executable
* Fix CI terminaterunfarm arg
* Add firesim.pem file to manager
* Bump python in CI instance
* Bump pip in CI container
* Remove pip sudo in CI container
* Fix launch script pip version equals
* Ini converted into strings
* Properly pass test_dir to opts in CI
* First pass at GH-A
* Round 2 CI fixes
* Try changes
* Remove CircleCI | Switch to fancy GH-A
* Rename self-host setup script
* Update chmod
* Use - instead of _ for env. vars
* Rename some defs | Remove extra imports
* Small comment updates
* Forgot to import in ini-api tests | Small comment on Fabric timeouts
* Add sys to linux poweroff
* Update linux timeout, fix small imports
* Update comment
* Fix-up workflow-monitor.py
* Avoid excessive logging in run-linux | Terminate spot instances after max-runtime
* Add more workflow-monitor states | Add pty=False to running workloads
* Update CI documentation | Add CI badge [ci skip]
* Don't use spot instances
* Update CI readme
* Determine runner version from remote repo and check for runner setup
* Address PR comments
* Update CI_README location of where to find IPs | Forgot ret_code
* Only run CI on prs/pushes to dev/main/master
* Fix terminate_workflow_instances in init-manager.py
* Cleanup FireSim repo cloning | Only run CI on PRs (since its runs on merge commit)
joey0320