firesim/regression/aws-ec2-tests/launch-setup-manager-instan...

45 lines
1.3 KiB
Bash
Raw Normal View History

Port to Python3 + Switch to GH-A CI (#878) * First pass at porting to python3 * Fix import errors | Setup user argcomplete * Update awstools CLI with user data file | Bump CI to use it * Wait until launch is complete * Add userdata as string | Use sudo for machine-launch-script * Remove execute permissions on machine-launch-script * Better match on machine-launch-script complete * Revert python-devel removal * Use python3 for pytests * Update more python3 items * Remove extra shebang * Port docs to python3 and add to CI * Add ISCA experiments to CI build check * Use yum not apt-get * Add make to doc section * Bump multilate-loadgen for sysroot fix * For BW test build don't use shebang * Fix docs Makefile options * Fix more doc warnings * Add first set of regression tests * Fix raw_input * Regression bump | Run workload fix * Add functools to topology * Fix linux poweroff test (nic still has issues) * Update regression scripts * Ignore machine-launch-script.sh in regression area * Fix map python3 issues * Get rid of shebangs * Fix more regressions * Print machine-launch.log on fail | More clarification on user_data * Transfer to CI some shorter regressions * Add a manual approval to fpga based tests * Fix indentation in config.yml * Fix test symlink * Use spot for CI manager instance | Try to use python3 for aws CI container | Version all pip packages * Make run-ini-api-tests an executable * Fix CI terminaterunfarm arg * Add firesim.pem file to manager * Bump python in CI instance * Bump pip in CI container * Remove pip sudo in CI container * Fix launch script pip version equals * Ini converted into strings * Properly pass test_dir to opts in CI * First pass at GH-A * Round 2 CI fixes * Try changes * Remove CircleCI | Switch to fancy GH-A * Rename self-host setup script * Update chmod * Use - instead of _ for env. vars * Rename some defs | Remove extra imports * Small comment updates * Forgot to import in ini-api tests | Small comment on Fabric timeouts * Add sys to linux poweroff * Update linux timeout, fix small imports * Update comment * Fix-up workflow-monitor.py * Avoid excessive logging in run-linux | Terminate spot instances after max-runtime * Add more workflow-monitor states | Add pty=False to running workloads * Update CI documentation | Add CI badge [ci skip] * Don't use spot instances * Update CI readme * Determine runner version from remote repo and check for runner setup * Address PR comments * Update CI_README location of where to find IPs | Forgot ret_code * Only run CI on prs/pushes to dev/main/master * Fix terminate_workflow_instances in init-manager.py * Cleanup FireSim repo cloning | Only run CI on PRs (since its runs on merge commit)
2022-01-21 03:03:37 +08:00
#!/bin/bash
set -ex
set -o pipefail
if [ $# -ne 1 ]; then
echo "$0 <FULL HASH TO TEST>"
exit 1
fi
SCRIPT_DIR="$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"
source $SCRIPT_DIR/defaults.sh
FULL_HASH=$1
# get the userdata file to launch manager with
rm -rf machine-launch-script.sh
wget https://raw.githubusercontent.com/firesim/firesim/$FULL_HASH/scripts/machine-launch-script.sh
# launch manager
$SCRIPT_DIR/../../deploy/awstools/awstools.py \
launch \
--inst_type c5.4xlarge \
--user_data_file $PWD/machine-launch-script.sh \
2>&1 | tee $IP_ADDR_FILE
rm -rf machine-launch-script.sh
# make sure managerinit finishes properly
2023-03-09 03:01:09 +08:00
run "timeout 10m grep -q \".*machine launch script complete.*\" <(tail -f /tmp/machine-launchstatus)"
Port to Python3 + Switch to GH-A CI (#878) * First pass at porting to python3 * Fix import errors | Setup user argcomplete * Update awstools CLI with user data file | Bump CI to use it * Wait until launch is complete * Add userdata as string | Use sudo for machine-launch-script * Remove execute permissions on machine-launch-script * Better match on machine-launch-script complete * Revert python-devel removal * Use python3 for pytests * Update more python3 items * Remove extra shebang * Port docs to python3 and add to CI * Add ISCA experiments to CI build check * Use yum not apt-get * Add make to doc section * Bump multilate-loadgen for sysroot fix * For BW test build don't use shebang * Fix docs Makefile options * Fix more doc warnings * Add first set of regression tests * Fix raw_input * Regression bump | Run workload fix * Add functools to topology * Fix linux poweroff test (nic still has issues) * Update regression scripts * Ignore machine-launch-script.sh in regression area * Fix map python3 issues * Get rid of shebangs * Fix more regressions * Print machine-launch.log on fail | More clarification on user_data * Transfer to CI some shorter regressions * Add a manual approval to fpga based tests * Fix indentation in config.yml * Fix test symlink * Use spot for CI manager instance | Try to use python3 for aws CI container | Version all pip packages * Make run-ini-api-tests an executable * Fix CI terminaterunfarm arg * Add firesim.pem file to manager * Bump python in CI instance * Bump pip in CI container * Remove pip sudo in CI container * Fix launch script pip version equals * Ini converted into strings * Properly pass test_dir to opts in CI * First pass at GH-A * Round 2 CI fixes * Try changes * Remove CircleCI | Switch to fancy GH-A * Rename self-host setup script * Update chmod * Use - instead of _ for env. vars * Rename some defs | Remove extra imports * Small comment updates * Forgot to import in ini-api tests | Small comment on Fabric timeouts * Add sys to linux poweroff * Update linux timeout, fix small imports * Update comment * Fix-up workflow-monitor.py * Avoid excessive logging in run-linux | Terminate spot instances after max-runtime * Add more workflow-monitor states | Add pty=False to running workloads * Update CI documentation | Add CI badge [ci skip] * Don't use spot instances * Update CI readme * Determine runner version from remote repo and check for runner setup * Address PR comments * Update CI_README location of where to find IPs | Forgot ret_code * Only run CI on prs/pushes to dev/main/master * Fix terminate_workflow_instances in init-manager.py * Cleanup FireSim repo cloning | Only run CI on PRs (since its runs on merge commit)
2022-01-21 03:03:37 +08:00
# setup the repo (similar to ci)
run "git clone https://github.com/firesim/firesim.git"
run "cd firesim/ && git checkout $FULL_HASH"
run "cd firesim/ && ./build-setup.sh --fast --skip-validate"
Port to Python3 + Switch to GH-A CI (#878) * First pass at porting to python3 * Fix import errors | Setup user argcomplete * Update awstools CLI with user data file | Bump CI to use it * Wait until launch is complete * Add userdata as string | Use sudo for machine-launch-script * Remove execute permissions on machine-launch-script * Better match on machine-launch-script complete * Revert python-devel removal * Use python3 for pytests * Update more python3 items * Remove extra shebang * Port docs to python3 and add to CI * Add ISCA experiments to CI build check * Use yum not apt-get * Add make to doc section * Bump multilate-loadgen for sysroot fix * For BW test build don't use shebang * Fix docs Makefile options * Fix more doc warnings * Add first set of regression tests * Fix raw_input * Regression bump | Run workload fix * Add functools to topology * Fix linux poweroff test (nic still has issues) * Update regression scripts * Ignore machine-launch-script.sh in regression area * Fix map python3 issues * Get rid of shebangs * Fix more regressions * Print machine-launch.log on fail | More clarification on user_data * Transfer to CI some shorter regressions * Add a manual approval to fpga based tests * Fix indentation in config.yml * Fix test symlink * Use spot for CI manager instance | Try to use python3 for aws CI container | Version all pip packages * Make run-ini-api-tests an executable * Fix CI terminaterunfarm arg * Add firesim.pem file to manager * Bump python in CI instance * Bump pip in CI container * Remove pip sudo in CI container * Fix launch script pip version equals * Ini converted into strings * Properly pass test_dir to opts in CI * First pass at GH-A * Round 2 CI fixes * Try changes * Remove CircleCI | Switch to fancy GH-A * Rename self-host setup script * Update chmod * Use - instead of _ for env. vars * Rename some defs | Remove extra imports * Small comment updates * Forgot to import in ini-api tests | Small comment on Fabric timeouts * Add sys to linux poweroff * Update linux timeout, fix small imports * Update comment * Fix-up workflow-monitor.py * Avoid excessive logging in run-linux | Terminate spot instances after max-runtime * Add more workflow-monitor states | Add pty=False to running workloads * Update CI documentation | Add CI badge [ci skip] * Don't use spot instances * Update CI readme * Determine runner version from remote repo and check for runner setup * Address PR comments * Update CI_README location of where to find IPs | Forgot ret_code * Only run CI on prs/pushes to dev/main/master * Fix terminate_workflow_instances in init-manager.py * Cleanup FireSim repo cloning | Only run CI on PRs (since its runs on merge commit)
2022-01-21 03:03:37 +08:00
run "cd firesim/sw/firesim-software && ./init-submodules.sh"
# use local aws permissions (for now bypass the manager)
copy ~/.aws/ $IP_ADDR:~/.aws
copy ~/firesim.pem $IP_ADDR:~/firesim.pem
copy firesim-managerinit.expect $IP_ADDR:~/firesim-managerinit.expect
run "cd firesim && source sourceme-manager.sh && cd ../ && ./firesim-managerinit.expect"
Port to Python3 + Switch to GH-A CI (#878) * First pass at porting to python3 * Fix import errors | Setup user argcomplete * Update awstools CLI with user data file | Bump CI to use it * Wait until launch is complete * Add userdata as string | Use sudo for machine-launch-script * Remove execute permissions on machine-launch-script * Better match on machine-launch-script complete * Revert python-devel removal * Use python3 for pytests * Update more python3 items * Remove extra shebang * Port docs to python3 and add to CI * Add ISCA experiments to CI build check * Use yum not apt-get * Add make to doc section * Bump multilate-loadgen for sysroot fix * For BW test build don't use shebang * Fix docs Makefile options * Fix more doc warnings * Add first set of regression tests * Fix raw_input * Regression bump | Run workload fix * Add functools to topology * Fix linux poweroff test (nic still has issues) * Update regression scripts * Ignore machine-launch-script.sh in regression area * Fix map python3 issues * Get rid of shebangs * Fix more regressions * Print machine-launch.log on fail | More clarification on user_data * Transfer to CI some shorter regressions * Add a manual approval to fpga based tests * Fix indentation in config.yml * Fix test symlink * Use spot for CI manager instance | Try to use python3 for aws CI container | Version all pip packages * Make run-ini-api-tests an executable * Fix CI terminaterunfarm arg * Add firesim.pem file to manager * Bump python in CI instance * Bump pip in CI container * Remove pip sudo in CI container * Fix launch script pip version equals * Ini converted into strings * Properly pass test_dir to opts in CI * First pass at GH-A * Round 2 CI fixes * Try changes * Remove CircleCI | Switch to fancy GH-A * Rename self-host setup script * Update chmod * Use - instead of _ for env. vars * Rename some defs | Remove extra imports * Small comment updates * Forgot to import in ini-api tests | Small comment on Fabric timeouts * Add sys to linux poweroff * Update linux timeout, fix small imports * Update comment * Fix-up workflow-monitor.py * Avoid excessive logging in run-linux | Terminate spot instances after max-runtime * Add more workflow-monitor states | Add pty=False to running workloads * Update CI documentation | Add CI badge [ci skip] * Don't use spot instances * Update CI readme * Determine runner version from remote repo and check for runner setup * Address PR comments * Update CI_README location of where to find IPs | Forgot ret_code * Only run CI on prs/pushes to dev/main/master * Fix terminate_workflow_instances in init-manager.py * Cleanup FireSim repo cloning | Only run CI on PRs (since its runs on merge commit)
2022-01-21 03:03:37 +08:00
echo "Success"