Adds a few metrics-based tools and a mailmap

This merge adds four new tools:

  * .mailmap : allows for easier identification of committers
  * tools/module_count.rb : Spits out a current count of modules
  * tools/module_commits.rb: Spits out who commited to a module
  * tools/committer_counts.rb : Spits out commiters by commit counts

This was part of a long-running feature branch, which is why it's now
bundled up in one big squash merge.

Squashed commit of the following:

commit de201ff6a5b304d0fedec56d9f1930abf1a10d9e
Author: Tod Beardsley <todb@metasploit.com>
Date:   Thu Jan 24 14:48:24 2013 -0600

    Rename from scorecard to merely a count

commit 8028cf838b0b560831602e3163e92d0751a4c0a9
Author: Tod Beardsley <todb@metasploit.com>
Date:   Thu Jan 24 14:36:42 2013 -0600

    Some final comment docs

commit a69fd7883837849664bc8777d119ac760de4a43d
Merge: e288f13 3faf4b3
Author: Tod Beardsley <todb@metasploit.com>
Date:   Thu Jan 24 13:21:14 2013 -0600

    Merge branch 'master' into committer-scorecard

    I think these conflicts came from a move or a rename or something.

    Conflicts:
    	external/source/exploits/cve-2012-5076_2/Makefile
    	external/source/exploits/cve-2012-5088/Makefile
    	modules/exploits/multi/browser/java_jre17_method_handle.rb
    	modules/exploits/multi/http/jenkins_script_console.rb

commit e288f13d7f7bca7aa4ceddd555b88d971a9f65a2
Author: Tod Beardsley <todb@metasploit.com>
Date:   Wed Jan 16 14:06:23 2013 -0600

    Add FireFart's mail alias

commit 1b1792e84febf015a79c3beb3d2473953da56935
Author: Tod Beardsley <todb@metasploit.com>
Date:   Fri Jan 18 22:41:44 2013 -0600

    Fix grammar on description for webcam

commit 276388fac541f0eebb9a18a980c5b474f438d117
Author: Robin Wood <robin@digininja.org>
Date:   Tue Jan 22 15:42:23 2013 +0000

    added extra checking for strict databases

commit a40ea3d73e52ab822cb89052ef7575f7ac52abb6
Author: jvazquez-r7 <juan.vazquez@metasploit.com>
Date:   Tue Jan 22 12:07:16 2013 +0100

    fix data added to table

commit 738d2fad5fccfbff23967ce219ad6bd4af90bbea
Author: HD Moore <hd_moore@rapid7.com>
Date:   Tue Jan 22 00:27:03 2013 -0600

    Fix a stack overflow in bidirectional pipe

commit aeec5a816b2f09f517930cdff074ea4b42ed5088
Author: jvazquez-r7 <juan.vazquez@metasploit.com>
Date:   Mon Jan 21 12:26:35 2013 +0100

    Cleanup for mysql_file_enum.rb

commit 13f68f089b4f3dd7c58bb4d5cb5767ff3df12852
Author: sinn3r <msfsinn3r@gmail.com>
Date:   Mon Jan 21 00:30:43 2013 -0600

    Updates the progress function

    Because the previous one was wrong.

commit d971fe0bb5f34667b6a621043838f7472e7255cd
Author: Robin Wood <robin@digininja.org>
Date:   Sun Jan 20 21:32:02 2013 +0000

    Brute force directory and file names with MySQL

commit a96ca2e96a3a34e302a6759ba48706c60b9724cd
Author: Robin Wood <robin@digininja.org>
Date:   Sun Jan 20 00:13:42 2013 +0000

    added a warning and using optpath

commit aa98d85abbc30166ce7d69a446bf78cddff92e0a
Author: Robin Wood <robin@digininja.org>
Date:   Sun Jan 20 00:12:38 2013 +0000

    added a warning and using optpath

commit 6dd5bb8532d0f68d44ca80099780428e0a3ad872
Author: Robin Wood <robin@digininja.org>
Date:   Sun Jan 20 00:02:07 2013 +0000

    stopped using fixed table name

commit 520aeb93119a77b4eb8d1187cac4084690d45613
Author: Robin Wood <robin@digininja.org>
Date:   Sat Jan 19 23:41:38 2013 +0000

    Fixed msftidy stuff

commit cec6a06c56444f12dc8b8985c2505b2d259d5077
Author: Robin Wood <robin@digininja.org>
Date:   Sat Jan 19 22:48:00 2013 +0000

    File/dir brute forcer using MySQL

commit 3cc0f3feaed87df11ab3695342af304d3b13d056
Author: jvazquez-r7 <juan.vazquez@metasploit.com>
Date:   Sun Jan 20 19:54:24 2013 +0100

    finally it doesn't use FileDropper atm

commit 2670d5ca8fbe2b26b2073445537bf0bfacd079dd
Author: jvazquez-r7 <juan.vazquez@metasploit.com>
Date:   Sun Jan 20 17:38:37 2013 +0100

    references and date updated

commit 1230d5267b3a8b33cfd64f6efb613986d6d13b31
Author: bcoles <bcoles@gmail.com>
Date:   Mon Jan 21 02:12:42 2013 +1030

    update php_charts_exec metadata

commit cf37c594e55b0130640f5aaea240b3aa936b7c8d
Author: bcoles <bcoles@gmail.com>
Date:   Mon Jan 21 02:10:48 2013 +1030

    move and update php_charts_exec metadata

commit 1e86429fa16a2f5d5003fbe6e69a74cac5efd767
Author: bcoles <bcoles@gmail.com>
Date:   Sun Jan 20 23:51:17 2013 +1030

    Add PHP-Charts v1.0 PHP Code Execution Exploit

commit fe60ee6dffc60a53b28bcfd08b5aada8bc8d4000
Author: jvazquez-r7 <juan.vazquez@metasploit.com>
Date:   Sun Jan 20 13:42:02 2013 +0100

    linux stager plus little cleanup

commit 5900248f585e7a5e10d93a0672aa8d330d5581ee
Author: Spencer McIntyre <zeroSteiner@gmail.com>
Date:   Sat Jan 19 19:10:56 2013 -0500

    use target_uri and normalize_uri as well as fix a cookie problem

commit a7ce0a500fe1ae4c71652191ee97ba1757cf65e0
Author: Spencer McIntyre <zeroSteiner@gmail.com>
Date:   Fri Jan 18 14:56:52 2013 -0500

    add module to execute commands via Jenkins Script Console

commit 33b8aa49f4dbbfbcc275b5cc0dfc43db9fec08f8
Author: jvazquez-r7 <juan.vazquez@metasploit.com>
Date:   Fri Jan 18 18:42:27 2013 +0100

    title updated

commit 63fe457fadf66ac27eac6210a26880c1f816d0ce
Author: Charles Smith <charles.smith@n2netsec.com>
Date:   Thu Jan 17 16:52:02 2013 -0500

    Fixed loot formatting so data is under the proper column

    The credentials table was defined with the columns "User", "Password", "Host", "Port", and "SSL".  Credentials were not added in that order, however. They were added in the order "host, port, user, password, ssl" in this line:

    credentials << [cred['host'], cred['port'], cred['user'], cred['password'], cred['ssl']]

    I changed the order the columns were defined to fix this.

    The permissions table had a similar issue. The "FileWrite" column was missing, so I added it. I also moved the "Home" column to after the "AutoCreate" column. Now the line:

    permissions << [perm['host'], perm['user'], perm['dir'], perm['fileread'], perm['filewrite'], perm['filedelete'], perm['fileappend'],perm['dircreate'], perm['dirdelete'], perm['dirlist'], perm['dirsubdirs'], perm['autocreate']]

    works correctly.

commit b948559b5ae0090c9ecb704bfba2da219577d4f4
Author: jvazquez-r7 <juan.vazquez@metasploit.com>
Date:   Thu Jan 17 21:45:13 2013 +0100

    cleanup

commit 199ab00a9c46295776b3f9c47d941721d5777a65
Author: jvazquez-r7 <juan.vazquez@metasploit.com>
Date:   Thu Jan 17 21:39:41 2013 +0100

    cleanup

commit 8d5504475dbce315581e87f395c9453bbe624d2e
Author: jvazquez-r7 <juan.vazquez@metasploit.com>
Date:   Thu Jan 17 21:27:47 2013 +0100

    Added new module for cve-2012-5076

commit 31ae18f392dea9fcfc4e1e6e1ec627aed2513d09
Author: jvazquez-r7 <juan.vazquez@metasploit.com>
Date:   Thu Jan 17 21:14:49 2013 +0100

    Added module for CVE-2012-5088

commit 6ac99f3db8f464767d15aaf60a2a5796b4ae8b30
Author: Tod Beardsley <todb@metasploit.com>
Date:   Sat Jan 19 09:08:31 2013 -0600

    Add a quick comment doc

commit 0c18f1c7cb53a77b4338e6014b76ea74749b41f9
Author: Tod Beardsley <todb@metasploit.com>
Date:   Sat Jan 19 09:06:34 2013 -0600

    Adds a per-module commit counter.

commit 44fa22832bb2e229f5a96a62658d7c4b0b88b966
Merge: fa288ff 9f42abd
Author: Tod Beardsley <todb@metasploit.com>
Date:   Sat Jan 19 08:30:37 2013 -0600

    Merge remote-tracking branch 'origin/master' into committer-scorecard

commit fa288ff007c1ead48ca011cda2488164d5103715
Author: Tod Beardsley <todb@metasploit.com>
Date:   Fri Jan 18 14:05:47 2013 -0600

    Make module_count execable

commit 6c1625ed709f505ec9e8be89820f9d6827a52567
Author: Tod Beardsley <todb@metasploit.com>
Date:   Tue Jan 8 09:56:48 2013 -0600

    Wrote a quick module counter, by type

commit af07ddc8184b85ecd43fb9e2cb2c607d54fb0c1b
Merge: 2ee5df8 2c3ccb5
Author: Tod Beardsley <todb@metasploit.com>
Date:   Tue Jan 8 09:35:28 2013 -0600

    Merge remote-tracking branch 'origin/master' into committer-scorecard

commit 2ee5df810313290a753344b83a9b9e591c30ef05
Merge: 501c678 b50e040
Author: Tod Beardsley <todb@metasploit.com>
Date:   Fri Jan 4 10:24:27 2013 -0600

    Merge remote-tracking branch 'origin/master' into committer-scorecard

commit 501c678b2ca6f67639d7d7425469d380ba6534cf
Merge: 8001401 c2586d0
Author: Tod Beardsley <todb@metasploit.com>
Date:   Thu Dec 27 15:42:25 2012 -0600

    Merge branch 'master' into committer-scorecard

commit 800140176686c8aa4e41629b259a1bcb8b7c9e0c
Author: Tod Beardsley <todb@metasploit.com>
Date:   Thu Dec 27 11:13:04 2012 -0600

    Adding shuckins and cjr to the mailmap

commit ab2db49c17b78616dc9199d62928e65d624e9e12
Merge: 8b6ecb3 daf5465
Author: Tod Beardsley <todb@metasploit.com>
Date:   Thu Dec 27 10:29:19 2012 -0600

    Merge remote branch 'origin/master' into committer-scorecard

commit 8b6ecb34bd2a1719bc51ab136cb9de1a8cd5c782
Author: Tod Beardsley <todb@metasploit.com>
Date:   Mon Dec 17 21:58:37 2012 -0600

    Comment docs on .mailmap

commit 8e245a086c2e91a80be31accdb6349837cba3dff
Author: Tod Beardsley <todb@metasploit.com>
Date:   Mon Dec 17 21:56:06 2012 -0600

    Another alias for h0ng10

commit aff6169602791a048cff2e41bac5cbb565abd341
Author: Tod Beardsley <todb@metasploit.com>
Date:   Mon Dec 17 17:02:35 2012 -0600

    A more useful committer score card

    Now with aliases for anyone who hit the top 20 list of the last year,
    six months, and twelve weeks.

    Still needs some optparsey niceities, but it's good enough for an
    intial push to GitHub.

commit bd4e00ee019cedfed2eb8af6b52786f5184193ca
Author: Tod Beardsley <todb@metasploit.com>
Date:   Mon Dec 17 15:22:33 2012 -0600

    Initial commit of a git commit scorecard
This commit is contained in:
Tod Beardsley 2013-01-24 14:56:28 -06:00
parent fbbac2bd51
commit 40fd695e6e
4 changed files with 246 additions and 0 deletions

63
.mailmap Normal file
View File

@ -0,0 +1,63 @@
wchen-r7 <wchen-r7@github> sinn3r <msfsinn3r@gmail.com> # aka sinn3r
wchen-r7 <wchen-r7@github> Wei Chen <Wei_Chen@rapid7.com>
wchen-r7 <wchen-r7@github> sinn3r <wei_chen@rapid7.com>
todb-r7 <todb-r7@github> Tod Beardsley <todb@metasploit.com>
jvazquez-r7 <jvazquez-r7@github> jvazquez-r7 <juan.vazquez@metasploit.com>
hmoore-r7 <hmoore-r7@github> HD Moore <hd_moore@rapid7.com>
jlee-r7 <jlee-r7@github> James Lee <egypt@metasploit.com> # aka egypt
jlee-r7 <jlee-r7@github> James Lee <James_Lee@rapid7.com>
dmaloney-r7 <dmaloney-r7@github> David Maloney <DMaloney@rapid7.com> # aka TheLightCosine
dmaloney-r7 <dmaloney-r7@github> David Maloney <David_Maloney@rapid7.com>
joev-r7 <joev-r7@github> Joe Vennix <Joe_Vennix@rapid7.com>
bturner-r7 <bturner-r7@github> Brandon Turner <brandon_turner@rapid7.com>
ecarey-r7 <ecarey-r7@github> Erran Carey <e@ipwnstuff.com>
limhoff-r7 <limhoff-r7@github> Luke Imhoff <luke_imhoff@rapid7.com>
todb-r7 <todb-r7@github> Tod Beardsley <tod_beardsley@rapid7.com>
tasos-r7 <tasos-r7@github> Tasos Laskos <Tasos_Laskos@rapid7.com>
shuckins-r7 <shuckins-r7@github> Samuel Huckins <samuel_huckins@rapid7.com>
# Above this line are current Rapid7 employees Below this paragraph are
# volunteers, former employees, and potential Rapid7 employees who, at
# one time or another, had some largeish number of commits landed on
# rapid7/metasploit-framework master branch. This should be refreshed
# periodically. If you're on this list and would like to not be, just
# let todb@metasploit.com know.
Meatballs1 <Meatballs1@github> Meatballs <eat_meatballs@hotmail.co.uk>
Meatballs1 <Meatballs1@github> Meatballs1 <eat_meatballs@hotmail.co.uk>
Meatballs1 <Meatballs1@github> Meatballs1 <campbell_ab@hotmail.com>
jduck <jduck@github> Joshua Drake <github.jdrake@qoop.org>
jcran <jcran@github> Jonathan Cran <jcran@rapid7.com>
jcran <jcran@github> Jonathan Cran <jcran@0x0e.org>
bannedit <bannedit@github> David Rude <bannedit0@gmail.com>
darkoperator <darkoperator@github> Carlos Perez <carlos_perez@darkoperator.com>
scriptjunkie <scriptjunkie@github> Matt Weeks <scriptjunkie@scriptjunkie.us>
skape <skape@???> Matt Miller <mmiller@hick.org>
ceballosm <ceballosm@github> Mario Ceballos <mc@metasploit.com>
swtornio <swtornio@github> Steve Tornio <swtornio@gmail.com>
kris <kris@???> kris <>
spoonm <spoonm@github> Spoon M <spoonm@gmail.com>
fab <fab@???> fab <> # fab at revhosts.net (Fabrice MOURRON)
efraintorres <efraintorres@github> efraintorres <etlownoise@gmail.com>
efraintorres <efraintorres@github> et <>
r3dy <r3dy@github> Royce Davis <r3dy@Royces-MacBook-Pro.local>
r3dy <r3dy@github> Royce Davis <royce.e.davis@gmail.com>
mubix <mubix@github> Rob Fuller <jd.mubix@gmail.com>
nmonkee <nmonkee@github> nmonkee <dave@northern-monkee.co.uk>
rsmudge <rsmudge@github> Raphael Mudge <rsmudge@gmail.com> # Aka `butane
m-1-k-3 <m-1-k-3@github> m-1-k-3 <michael.messner@integralis.com>
kost <kost@github> Vlatko Kosturjak <kost@linux.hr>
ohdae <ohdae@github> ohdae <bindshell@live.com>
schierlm <schierlm@github> Michael Schierl <schierlm@gmx.de> # Aka mihi
corelanc0d3er <corelanc0d3er@github> Peter Van Eeckhoutte (corelanc0d3r) <peter.ve@corelan.be>
corelanc0d3er <corelanc0d3er@github> corelanc0d3r <peter.ve@corelan.be>
kernelsmith <kernelsmith@github> Joshua Smith <kernelsmith@kernelsmith.com>
kernelsmith <kernelsmith@github> kernelsmith <kernelsmith@kernelsmith>
h0ng10 <h0ng10@github> h0ng10 <hansmartin.muench@googlemail.com>
h0ng10 <h0ng10@github> Hans-Martin Münch <hansmartin.muench@googlemail.com>
nullbind <nullbind@github> nullbind <scott.sutherland@nullbind.com>
nevdull77 <nevdull77@github> Patrik Karlsson <patrik@cqure.net>
jgor <jgor@github> jgor <jgor@indiecom.org>
ChrisJohnRiley <ChrisJohnRiley@github> Chris John Riley <chris.riley@c22.cc>
ChrisJohnRiley <ChrisJohnRiley@github> Chris John Riley <reg@c22.cc>
FireFart <FireFart@github> Christian Mehlmauer <firefart@gmail.com>

70
tools/committer_count.rb Executable file
View File

@ -0,0 +1,70 @@
#!/usr/bin/env ruby
# The commit_scorecard is a way to tell who's been active over the last
# given period. It's of course, quite coarse -- someone with 10 commits in a day
# may or may not be more productive than someone with 3, but over long enough
# periods, it's an okay metric to measure involvement with the project, since
# large and small commits will tend to average out.
#
# Note that this includes merge commits by default (which usually means at least
# code review happened, so it's still a measure of work). You can get different
# stats by ignoring merge commits, once option parsing is implemented.
#
# Usage: ./commit_scorecard.rb 2011-01-01 | head -10 # Since a particular date
# ./commit_scorecard.rb 1y | head -10 # Last year
# ./commit_scorecard.rb 6m | head -10 # Last six months
# ./commit_scorecard.rb 12w | head -10 # Last twelve weeks
# ./commit_scorecard.rb 100d | head -10 # Last hundred days
#
#
# History with colors and e-mail addresses (respecting .mailmap):
# git log --pretty=format:"%C(white)%ad %C(yellow)%h %Cblue'%aN' <%aE> %Cgreen%f%Creset" --date=short
class GitLogLine < Struct.new(:date, :hash, :author, :message)
end
@history = `git log --pretty=format:"%ad %h '%aN' %f" --date=short --date-order`
@recent_history = []
@commits_by_author = {}
def parse_date(date)
case date
when /([0-9]+)y(ear)?s?/
seconds = $1.to_i* (60*60*24*365.25)
calc_date = (Time.now - seconds).strftime("%Y-%m-%d")
when /([0-9]+)m(onth)?s?/
seconds = $1.to_i* (60*60*24*(365.25 / 12))
calc_date = (Time.now - seconds).strftime("%Y-%m-%d")
when /([0-9]+)w(eek)?s?/
seconds = $1.to_i* (60*60*24*7)
calc_date = (Time.now - seconds).strftime("%Y-%m-%d")
when /([0-9]+)d(ay)?s?/
seconds = $1.to_i* (60*60*24)
calc_date = (Time.now - seconds).strftime("%Y-%m-%d")
else
calc_date = Time.new(date).strftime("%Y-%m-%d")
end
end
date = ARGV[0] || "2005-03-22" # A day before the first SVN commit.
calc_date = parse_date(date)
@history.each_line do |line|
parsed_line = line.match(/^([^\s+]+)\s(.{7,})\s'(.*)'\s(.*)[\r\n]*$/)
break if calc_date == parsed_line[1]
@recent_history << GitLogLine.new(*parsed_line[1,4])
end
@recent_history.each do |logline|
@commits_by_author[logline.author] ||= []
@commits_by_author[logline.author] << logline.message
end
puts "Commits since #{calc_date}"
puts "-" * 50
@commits_by_author.sort_by {|k,v| v.size}.reverse.each do |k,v|
puts "%-25s %3d" % [k,v.size]
end

64
tools/module_commits.rb Executable file
View File

@ -0,0 +1,64 @@
#!/usr/bin/env ruby
# Check the commit history of a module or tree of modules.
# and sort by number of commits.
#
# Usage: tools/module_commits.rb [module dir | module fname]
require 'find'
class GitLogLine < Struct.new(:date, :hash, :author, :message)
end
class CommitHistory < Struct.new(:fname, :total, :authors)
end
msfbase = __FILE__
while File.symlink?(msfbase)
msfbase = File.expand_path(File.readlink(msfbase), File.dirname(msfbase))
end
dir = ARGV[0] || File.join(msfbase, "modules", "exploits")
raise ArgumentError, "Need a filename or directory" unless (dir and File.readable? dir)
def check_commit_history(fname)
git_cmd = `git log --pretty=format:"%ad %h '%aN' %f" --date=short --date-order #{fname}`
commit_history = []
commits_by_author = {}
git_cmd.each_line do |line|
parsed_line = line.match(/^([^\s+]+)\s(.{7,})\s'(.*)'\s(.*)[\r\n]*$/)
commit_history << GitLogLine.new(*parsed_line[1,4])
end
commit_history.each do |logline|
commits_by_author[logline.author] ||= []
commits_by_author[logline.author] << logline.message
end
puts "Commits for #{fname} #{commit_history.size}"
puts "-" * 72
commits_by_author.sort_by {|k,v| v.size}.reverse.each do |k,v|
puts "%-25s %3d" % [k,v.size]
end
this_module = CommitHistory.new(fname,commit_history.size,commits_by_author)
return this_module
end
@module_stats = []
Find.find(dir) do |fname|
next unless fname =~ /rb$/
@module_stats << check_commit_history(fname)
end
puts "=" * 72
puts "Sorted modules by commit counts"
@module_stats.sort_by {|m| m.total }.reverse.each do |m|
puts "%-60s %d" % [m.fname, m.total]
end

49
tools/module_count.rb Executable file
View File

@ -0,0 +1,49 @@
#!/usr/bin/env ruby
# Lists the current count of modules, by type, and outputs a bare CSV.
msfbase = __FILE__
while File.symlink?(msfbase)
msfbase = File.expand_path(File.readlink(msfbase), File.dirname(msfbase))
end
$:.unshift(File.expand_path(File.join(File.dirname(msfbase), '..', 'lib')))
require 'fastlib'
require 'msfenv'
$:.unshift(ENV['MSF_LOCAL_LIB']) if ENV['MSF_LOCAL_LIB']
require 'rex'
require 'msf/ui'
require 'msf/base'
# Always disable the database (we never need it just to list module
# information).
framework_opts = { 'DisableDatabase' => true }
# Initialize the simplified framework instance.
$framework = Msf::Simple::Framework.create(framework_opts)
Indent = ' '
i = 0
module_types = {
:exploit => 0,
:auxiliary => 0,
:post => 0,
:payload => 0,
:encoder => 0,
:nop => 0
}
$framework.modules.each do |name, mod|
this_mod = mod.new
[:exploit, :auxiliary, :post, :payload, :encoder, :nop].each do |meth|
interrogative = "#{meth}?".intern
if this_mod.send(interrogative)
module_types[meth] += 1
end
end
end
puts module_types.keys.map {|k| k.to_s}.join(",")
puts module_types.values.join(",")