Commit Graph

56 Commits

Author SHA1 Message Date
Luke Imhoff 633b4b9e51
Disable simplecov on travis-ci
MSP-11671

`bundler install --without coverage` will not install `simplecov`.
Running without simplecov changes the average runtime of `rake
cucumber:boot` from (n=13) 112.50 seconds to (n=10) 32.17 seconds
(-71.41%). (-73.68% from 2c1203ba49d49603aec169eed053ab929ccdedc6.)  Raw
data:
https://docs.google.com/spreadsheets/d/1vCRi_38Go3Wfq670eDCXMTSSXL1blSbk6NQsgpOqfYw/edit?usp=sharing.
2014-12-05 11:58:09 -06:00
Luke Imhoff 08a8cee08a
Don't require simplecov in cucumber env.rb
MSP-11671

All features currently use aruba to run msfconsole in a separate process
and it is that child process that needs to load simplecov.  Changes
average runtime for `rake cucumber:boot` from (n=10) 122.21 seconds to
(n=13) 112.50 seconds (-7.50%).  Raw data:
https://docs.google.com/spreadsheets/d/1vCRi_38Go3Wfq670eDCXMTSSXL1blSbk6NQsgpOqfYw/edit?usp=sharing
2014-12-05 10:21:58 -06:00
Luke Imhoff 2c1203ba49
"Starting `msfconsole` with a valid database.yml" --defer-module-loads
MSP-11671

Run `msfconsole` with `--defer-module-loads` to change average (n=10)
runtime of `/usr/bin/time rake CUCUMBER_OPTS="-r features"
FEATURE=features/msfconsole/database_yml.feature:150 cucumber:boot` from 57.02
seconds to 38.76 seconds (-32.03%).  (-80.26% change from
288d77f31a14f4d05c7f665c2f8bbfd0c58e8d4d.) Raw data:
https://docs.google.com/spreadsheets/d/1vCRi_38Go3Wfq670eDCXMTSSXL1blSbk6NQsgpOqfYw/edit?usp=sharing.
2014-12-04 13:28:21 -06:00
Luke Imhoff 7dfdc508a7
"Starting `msfconsole` with a valid database.yml" --execute-command
MSP-11671

Run `msfconsole` with `--execute-command db_status --execute-command
exit` instead of running interactively and typing `db_status` and `exit`
to change the average (n=10) running time of `/usr/bin/time rake
CUCUMBER_OPTS="-r features"
FEATURE=features/msfconsole/database_yml.feature:150 cucumber:boot` from
196.30 seconds to 57.02 seconds (-70.95%).  Raw data:
https://docs.google.com/spreadsheets/d/1vCRi_38Go3Wfq670eDCXMTSSXL1blSbk6NQsgpOqfYw/edit?usp=sharing.
2014-12-04 12:58:37 -06:00
Luke Imhoff 288d77f31a
"no database connection" --defer-module-loads
MSP-11671

Call `msfconsole` with `--defer-module-loads` to change runtime of
`/usr/bin/time rake CUCUMBER_OPTS="-r features"
FEATURE=features/msfconsole/database_yml.feature:134 cucumber:boot` from
49.44 seconds to 37.65 seconds (-23.83%). (-80.73% change from
22c56f525335d45623c7708ccc53561e516ae8a0.)  Raw data:
https://docs.google.com/spreadsheets/d/1vCRi_38Go3Wfq670eDCXMTSSXL1blSbk6NQsgpOqfYw/edit?usp=sharing.
2014-12-04 11:39:41 -06:00
Luke Imhoff ec63f655c8
"no database connection" --execute-command
MSP-11671

Call `msfconsole` with `--execute-command db_status --execute_command
exit` instead of running interactively and typing `db_status` and `exit`
to change the average (n=10) run time of `/usr/bin/time rake
CUCUMBER_OPTS="-r features"
FEATURE=features/msfconsole/database_yml.feature:134 cucumber:boot` from 195.38
seconds to 49.44 seconds (-74.70%).
2014-12-04 11:26:49 -06:00
Luke Imhoff 22c56f5253
`Without --yaml, MSF_DATABASE_CONFIG or ~/.msf4/database.yml, project "database.yml" wins` --defer-module-loads
MSP-11671

Run `msfconsole` with `--defer-module-loads` to change average (n=10)
runtime of `/usr/bin/time rake CUCUMBER_OPTS="-r features"
FEATURE=features/msfconsole/database_yml.feature:115 cucumber:boot` from
51.42 seconds to 37.76 seconds (-26.57%).  (Change of -72.98% since
f950f3094de718f5294a84fb7e0a78390818ba4d.)  Raw data:
https://docs.google.com/spreadsheets/d/1vCRi_38Go3Wfq670eDCXMTSSXL1blSbk6NQsgpOqfYw/edit?usp=sharing.
2014-12-04 09:48:07 -06:00
Luke Imhoff 7a347accd8
`Without --yaml, MSF_DATABASE_CONFIG or ~/.msf4/database.yml, project "database.yml" wins` --execute-command
MSP-11671

Run msfconsole with `--execute-command exit` instead of running
interactly and typing `exit`.  Changes average (n=10) running time for
`/usr/bin/time rake CUCUMBER_OPTS="-r features"
FEATURE=features/msfconsole/database_yml.feature:115 cucumber:boot` from 139.76
seconds to 51.42 seconds (-63.21%).
2014-12-04 09:33:53 -06:00
Luke Imhoff f950f3094d
"Without --yaml or MSF_DATABASE_CONFIG, ~/.msf4/database.yml wins" --defer-module-loads
MSP-11671

Pass `--defer-module-loads` to `msfconsole` to change average (n=10)
runtime for `/usr/bin/time rake CUCUMBER_OPTS="-r features"
FEATURE=features/msfconsole/database_yml.feature:88 cucumber:boot` from
51.79 seconds to 39.25 seconds (-24.40%). (-71.74% from
e9a69a844cd2f2f7ad24386dd3adf7ccbd18d2b3.) Raw data:
https://docs.google.com/spreadsheets/d/1vCRi_38Go3Wfq670eDCXMTSSXL1blSbk6NQsgpOqfYw/edit?usp=sharing.
2014-12-03 15:44:04 -06:00
Luke Imhoff 186abd5e05
"Without --yaml or MSF_DATABASE_CONFIG, ~/.msf4/database.yml wins" --execute-command
MSP-11671

Run `msfconsole` with `--execute-command exit` instead of interactively
and typing exit.  Average (n=10) runtime changes from 138.91 seconds to
51.79 seconds (-62.72%). Raw data:
https://docs.google.com/spreadsheets/d/1vCRi_38Go3Wfq670eDCXMTSSXL1blSbk6NQsgpOqfYw/edit?usp=sharing.
2014-12-03 15:30:41 -06:00
Luke Imhoff e9a69a844c
"Without --yaml, MSF_DATABASE_CONFIG wins" --defer-module-loads
MSP-11671

Add `--defer-module-loads` when running `msfconsole`.  Changes average
(n=10) runtime from 52.96 seconds to 38.98 seconds (-26.39%) (-71.92%
compared to 69b170a7df).
2014-12-03 14:26:58 -06:00
Luke Imhoff 8b88d7bce7
"Without --yaml, MSF_DATABASE_CONFIG wins" --execute-command
MSP-11671

Pass `--execute-command exit` to `msfconsole` instead of running it
interactively and typing `exit`.  Average (n=10) runtime for `rake
CUCUMBER_OPTS="-r features"
FEATURE=features/msfconsole/database_yml.feature:54 cucumber:boot`
change from 138.84 seconds to 52.96 seconds (-61.85%).
2014-12-03 14:12:12 -06:00
Luke Imhoff 69b170a7df
"With all 4 locations, --yaml wins" --defer-module-loads
MSP-11671

Add --defer-module-loads to msfconsole run to reduce average (n=10)
running time from 51.78 seconds to 40.38 seconds (-22.02% change).  This
is a -71.55% change since 4eb9a7563a.  Raw
data:
https://docs.google.com/spreadsheets/d/1vCRi_38Go3Wfq670eDCXMTSSXL1blSbk6NQsgpOqfYw/edit?usp=sharing.
2014-12-03 12:54:21 -06:00
Luke Imhoff 0e1e5125c6
"With all 4 locations, --yaml wins" --execute-command
MSP-11671

Use `--execute-command` instead of interactly running msfconsole and
typing `exit`.  Changes the average runtime for `rake CUCUMBER_OPTS="-r
features" FEATURE=features/msfconsole/database_yml.feature:13
cucumber:boot` from 141.93 seconds (n=10) to 51.78 seconds (n=10) for a
-63.52% change. Raw data:
https://docs.google.com/spreadsheets/d/1vCRi_38Go3Wfq670eDCXMTSSXL1blSbk6NQsgpOqfYw/edit?usp=sharing.
2014-12-03 12:38:43 -06:00
Luke Imhoff 4eb9a7563a
Use -x when testing help
MSP-11671

Instead of interactively typing `help` and `exit` use `-x` to run the
commands, which changes the average (n=10) time for `rake cucumber` from
153.29 seconds to an average (n=10) of 32.21 for a -78.99% change from
the last commit and a -80.47% change from the beginning of this branch
(3c3792330c).  Raw data:
https://docs.google.com/spreadsheets/d/1vCRi_38Go3Wfq670eDCXMTSSXL1blSbk6NQsgpOqfYw/edit?usp=sharing
2014-12-03 10:23:03 -06:00
Luke Imhoff 73844ebdd9
--defer-module-loads when testing help
MSP-11671

Changed runtime of `rake cucumber` from average (n=10) of 164.90 seconds to
average (n=10) of 153.29 seconds for a -7.04% decrease.  Raw data:
https://docs.google.com/spreadsheets/d/1vCRi_38Go3Wfq670eDCXMTSSXL1blSbk6NQsgpOqfYw/edit?usp=sharing
2014-12-03 08:44:13 -06:00
Luke Imhoff 3c3792330c
Fix RUBYOPT to be debugger compatible
MSP-11671

RUBYOPT needs to be appended to so that debugger is always first.
2014-12-02 15:31:57 -06:00
Luke Imhoff 48356d1003 Up aruba timeout for simplecov overhead
MSP-11647

Doubling the timeout is arbitrary, but travis-ci's limit for no output
is 10 minutes, so it can't go much higher.
2014-11-25 14:13:41 -06:00
Luke Imhoff 81d160c685 Add simplecov to cucumber env
MSP-11647

Port simplecov setup from metasploit-version so that child processes, in
this case msfconsole, will load simplecov and report coverage back to
the parent process.  Coverage from msfconsole's cucumber features is the
only way to get coverage on portions of the command dispatcher commands.
2014-11-25 14:13:40 -06:00
James Lee d86d8e9092
Remove the now-unnecessary -L 2014-10-30 15:36:16 -05:00
James Lee 4fb58ebba2
Force msfconsole to use stdlib's readline with -L 2014-10-29 13:21:31 -05:00
darkbushido 835f224b3c
tagging the module tests as WIP since proxies keep showing up in options 2014-09-16 14:35:57 -05:00
darkbushido 090338f575
changing the cucumber timeout to 4 minutes.... 2014-09-11 16:54:19 -05:00
darkbushido 4ef60d1a00
removing unused step files
changing the ms08_067 tests to fail when proxies show up in the wrong section
2014-09-11 16:51:49 -05:00
darkbushido 50e804d9cb
updating how we use cucumber to match lukes changes 2014-09-02 18:57:55 -05:00
darkbushido 5b3800c13a
removing the hacked stty solution 2014-09-02 13:57:48 -05:00
darkbushido 13a0b8da83
Merge branch 'master' into aruba-testing-with-updated-master
Conflicts:
	Gemfile.lock
	config/cucumber.yml
	features/support/env.rb
	features/support/hooks.rb
	lib/tasks/cucumber.rake
2014-09-02 13:56:02 -05:00
Luke Imhoff 20177c7c23
Restore backup database.yml when retesting after interrupt
MSP-11153

Restore the config/database.yml backed up to
config/database.yml.cucumber.bak in the db:config:restore task, which is
made a dependency of the environment rake task so that
config/database.yml is restored before Rails tries to use it in the
environment task.  This specifically, allows for rake cucumber to be
interrupted when the config/database.yml has been moved to
config/database.yml.cucumber.bak and a subsequence rake cucumber to
succeed and restore config/database.yml, but any task that depends on
environment will restore the config/database.yml.
2014-08-28 15:20:53 -05:00
Luke Imhoff 188f5d012a
Add scenario for no database.yml
MSP-11153
2014-08-27 22:02:16 -05:00
Luke Imhoff 7453f6fa3a
Project "database.yml" scenario
MSP-11153
2014-08-27 21:47:31 -05:00
Luke Imhoff e6750b985c
Add 'the' to make steps read better
MSP-11153
2014-08-27 21:38:46 -05:00
Luke Imhoff 972470c241
Ensure a fake project database.yml is used for scenarios
MSP-11153

Ensures that cucumber still works if config/database.yml is not set and
so other location is being used to run cucumber.
2014-08-27 21:36:23 -05:00
Luke Imhoff 496865e591
Order database.yml definitions to match precedence
MSP-11153
2014-08-27 21:15:00 -05:00
Luke Imhoff d752cdccf6
Remove unneeded command_line.yml
MSP-11153

Remove definition of command_line.yml in scenarios that don't use --yaml
flag.
2014-08-27 21:09:49 -05:00
Luke Imhoff b701ba5dcf
~/.msf4/database.yml scenario
MSP-11153
2014-08-27 20:57:08 -05:00
Luke Imhoff b1e745aa16
MSF_DATABASE_CONFIG scenario
MSP-11153
2014-08-27 20:33:52 -05:00
Luke Imhoff bfc509c18a
Add feature that tests --yaml is favored over others
MSP-11153
2014-08-27 16:46:23 -05:00
Luke Imhoff 1857c6ae39
Add aruba
MSP-11153

aruba adds steps for testing commandline applications with cucumber.
2014-08-27 14:22:20 -05:00
Luke Imhoff 2f48f7c48c
rails generate cucumber:install
MSP-11153

Add cucumber-rails for testing msfconsole's loading of database.yml from
different paths.
2014-08-27 14:10:04 -05:00
darkbushido 984f073c7d
changing from cucumber to cucumber-rails
using the MSF_DATABASE_CONFIG to tell msfconsole where to find the database.yml
2014-08-27 13:44:34 -05:00
darkbushido df62c4a29d
setting the timeout to 180 2014-08-21 10:15:07 -05:00
darkbushido 264192e796
setting the timeout to 120 2014-08-21 10:00:35 -05:00
darkbushido 9e4b02abb1
increasing the aruba timeout to 240 seconds
msfconsole is taking over 60 seconds to load on travis
2014-08-21 09:45:53 -05:00
darkbushido 73911e9007
adding in the positive test case 2014-08-20 15:54:49 -05:00
darkbushido 15a7ed0031
adding scenarios for booting msfconsole without a db 2014-08-20 11:02:11 -05:00
darkbushido f80fd68480
adding more scenarios
updating ms08-067's scenarios so they are testing show options and show advanced
changing the scenario descriptions and feature descriptions for help.feature and
ms08-067
2014-08-19 15:06:47 -05:00
darkbushido 8f3f37dd54
updating how cucumber loads features
starting to use a cucumber.yml to store the profile configs
we will not be running @targets by default
2014-08-19 15:05:47 -05:00
darkbushido 565f622fe0
renaming targets.yml to .example
ignoring the targets.yml file
2014-08-13 17:40:51 -05:00
darkbushido 0d045d007a
moving the module file
adding some comments on things we should not see when running
msfconsole
2014-08-13 17:37:40 -05:00
Jon Hart e66d2c8d19 Merge branch 'aruba-testing' into target_hooks 2014-08-13 15:29:53 -07:00