Module: Metasploit::Framework::Spec::Threads::Suite

Defined in:
lib/metasploit/framework/spec/threads/suite.rb

Constant Summary collapse

EXPECTED_THREAD_COUNT_AROUND_SUITE =

Number of allowed threads when threads are counted in ‘after(:suite)` or `before(:suite)`

Known threads:

1. Main Ruby thread
2. Active Record connection pool thread
3. Framework thread manager, a monitor thread for removing dead threads
   https://github.com/rapid7/metasploit-framework/blame/04e8752b9b74cbaad7cb0ea6129c90e3172580a2/lib/msf/core/thread_manager.rb#L66-L89
4. Ruby's Timeout library thread, an automatically created monitor thread when using `Thread.timeout(1) { }`
   https://github.com/ruby/timeout/blob/bd25f4b138b86ef076e6d9d7374b159fffe5e4e9/lib/timeout.rb#L129-L137
5. REMOTE_DB thread, if enabled

Intermittent threads that are non-deterministically left behind, which should be fixed in the future:

1. metadata cache hydration
   https://github.com/rapid7/metasploit-framework/blob/115946cd06faccac654e956e8ba9cf72ff328201/lib/msf/core/modules/metadata/cache.rb#L150-L153
2. session manager
   https://github.com/rapid7/metasploit-framework/blob/115946cd06faccac654e956e8ba9cf72ff328201/lib/msf/core/session_manager.rb#L153-L168

ENV['REMOTE_DB'] ? 7 : 6
LOG_PATHNAME =

‘caller` for all Thread.new calls

Pathname.new('log/metasploit/framework/spec/threads/suite.log')
UUID_REGEXP =

Regular expression for extracting the UUID out of LOG_PATHNAME for each Thread.new caller block

/BEGIN Thread.new caller \((?<uuid>.*)\)/
UUID_THREAD_LOCAL_VARIABLE =

Name of thread local variable that Thread UUID is stored

"metasploit/framework/spec/threads/logger/uuid"

Class Method Summary collapse

Class Method Details

.caller_by_thread_uuid ⇒ Hash{String => Array<String>}

The ‘caller` for each Thread UUID.

Returns:

  • (Hash{String => Array<String>}) 


224
225
226
227
228
229
230
231
232
233
234
 
# File 'lib/metasploit/framework/spec/threads/suite.rb', line 224

def self.caller_by_thread_uuid
  lines_by_thread_uuid = Hash.new { |hash, uuid|
    hash[uuid] = []
  }

  each_thread_line do |uuid, line|
    lines_by_thread_uuid[uuid] << line
  end

  lines_by_thread_uuid
end

.configure! ⇒ void

This method returns an undefined value.

Configures ‘before(:suite)` and `after(:suite)` callback to detect thread leaks.



47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
 
# File 'lib/metasploit/framework/spec/threads/suite.rb', line 47

def self.configure!
  unless @configured
    RSpec.configure do |config|
      config.before(:suite) do
        thread_count = Metasploit::Framework::Spec::Threads::Suite.non_debugger_thread_list.count

        # check with if first so that error message can be constructed lazily
        if thread_count > EXPECTED_THREAD_COUNT_AROUND_SUITE
          # LOG_PATHNAME may not exist if suite run without `rake spec`
          if LOG_PATHNAME.exist?
            log = LOG_PATHNAME.read()
          else
            log "Run `rake spec` to log where Thread.new is called."
          end

          raise RuntimeError,
                "#{thread_count} #{'thread'.pluralize(thread_count)} exist(s) when " \
                "only #{EXPECTED_THREAD_COUNT_AROUND_SUITE} " \
                "#{'thread'.pluralize(EXPECTED_THREAD_COUNT_AROUND_SUITE)} expected before suite runs:\n" \
                "#{log}"
        end

        LOG_PATHNAME.parent.mkpath

        LOG_PATHNAME.open('a') do |f|
          # separator so after(:suite) can differentiate between threads created before(:suite) and during the
          # suites
          f.puts 'before(:suite)'
        end
      end

      config.after(:suite) do
        LOG_PATHNAME.parent.mkpath

        LOG_PATHNAME.open('a') do |f|
          # separator so that a flip flop can be used when reading the file below.  Also useful if it turns
          # out any threads are being created after this callback, which could be the case if another
          # after(:suite) accidentally created threads by creating an Msf::Simple::Framework instance.
          f.puts 'after(:suite)'
        end

        thread_list = Metasploit::Framework::Spec::Threads::Suite.non_debugger_thread_list
        thread_count = thread_list.count

        if thread_count > EXPECTED_THREAD_COUNT_AROUND_SUITE
          error_lines = []

          caller_by_thread_uuid = if LOG_PATHNAME.exist?
                                    Metasploit::Framework::Spec::Threads::Suite.caller_by_thread_uuid
                                  else
                                    error_lines << "Note: #{LOG_PATHNAME} does not exist. Run `rake spec` to log where Thread.new is called.\n\n"
                                    {}
                                  end

          thread_list.each_with_index do |thread, index|
            thread_uuid = thread[Metasploit::Framework::Spec::Threads::Suite::UUID_THREAD_LOCAL_VARIABLE]
            thread_name = thread[:tm_name]

            error_lines << "--- Thread #{index + 1}/#{thread_count} ---\n"
            error_lines << "  UUID: #{thread_uuid || '(none - unmanaged thread)'}\n"
            error_lines << "  Name: #{thread_name.inspect}\n"
            error_lines << "  Status: #{thread.status.inspect}\n"
            error_lines << "  Class: #{thread.class.name}\n"
            error_lines << "  Priority: #{thread.priority}\n"
            error_lines << "  Thread locals: #{thread.keys.map(&:to_s).sort.inspect}\n"

            if thread_uuid && caller_by_thread_uuid[thread_uuid]
              error_lines << "  Creation caller:\n"
              caller_by_thread_uuid[thread_uuid].each do |caller_line|
                error_lines << "    #{caller_line}"
              end
            end

            backtrace = thread.backtrace
            if backtrace
              error_lines << "  Current backtrace:\n"
              backtrace.each do |bt_line|
                error_lines << "    #{bt_line}\n"
              end
            else
              error_lines << "  Current backtrace: nil (no backtrace available)\n"
            end

            error_lines << "\n"
          end

          raise RuntimeError,
                "#{thread_count} #{'thread'.pluralize(thread_count)} exist(s) when only " \
                "#{EXPECTED_THREAD_COUNT_AROUND_SUITE} " \
                "#{'thread'.pluralize(EXPECTED_THREAD_COUNT_AROUND_SUITE)} expected after suite runs:\n" \
                "#{error_lines.join}"
        end
      end
    end

    @configured = true
  end

  @configured
end

.define_task ⇒ Object 



148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
 
# File 'lib/metasploit/framework/spec/threads/suite.rb', line 148

def self.define_task
  Rake::Task.define_task('metasploit:framework:spec:threads:suite') do
    if Metasploit::Framework::Spec::Threads::Suite::LOG_PATHNAME.exist?
      Metasploit::Framework::Spec::Threads::Suite::LOG_PATHNAME.delete
    end

    parent_pathname = Pathname.new(__FILE__).parent
    threads_logger_pathname = parent_pathname.join('logger')
    load_pathname = parent_pathname.parent.parent.parent.parent.expand_path

    # Must append to RUBYOPT or Rubymine debugger will not work
    ENV['RUBYOPT'] = "#{ENV['RUBYOPT']} -I#{load_pathname} -r#{threads_logger_pathname}"
  end

  Rake::Task.define_task(spec: 'metasploit:framework:spec:threads:suite')
end

.each_suite_line {|line| ... } ⇒ Object

Note:

Ensure LOG_PATHNAME exists before calling.

Yields each line of LOG_PATHNAME that happened during the suite run.

Yields:

  • (line)

Yield Parameters:

  • line (String) —

    a line in the LOG_PATHNAME between ‘before(:suite)` and `after(:suite)`

Yield Returns:

  • (void) 


172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
 
# File 'lib/metasploit/framework/spec/threads/suite.rb', line 172

def self.each_suite_line
  in_suite = false

  LOG_PATHNAME.each_line do |line|
    if in_suite
      if line.start_with?('after(:suite)')
        break
      else
        yield line
      end
    else
      if line.start_with?('before(:suite)')
        in_suite = true
      end
    end
  end
end

.each_thread_line {|uuid, line| ... } ⇒ Object

Note:

Ensure LOG_PATHNAME exists before calling.

Yield each line for each Thread UUID gathered during the suite run.

Yields:

  • (uuid, line)

Yield Parameters:

  • uuid (String) —

    the UUID of thread thread

  • line (String) —

    a line in the ‘caller` for the given `uuid`

Yield Returns:

  • (void) 


198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
 
# File 'lib/metasploit/framework/spec/threads/suite.rb', line 198

def self.each_thread_line
  in_thread_caller = false
  uuid = nil

  each_suite_line do |line|
    if in_thread_caller
      if line.start_with?('END Thread.new caller')
        in_thread_caller = false
        next
      else
        yield uuid, line
      end
    else
      match = line.match(UUID_REGEXP)

      if match
        in_thread_caller = true
        uuid = match[:uuid]
      end
    end
  end
end

.non_debugger_thread_list ⇒ Object

Returns:



237
238
239
240
241
242
243
244
 
# File 'lib/metasploit/framework/spec/threads/suite.rb', line 237

def self.non_debugger_thread_list
  Thread.list.reject { |thread|
    # don't do `is_a? Debugger::DebugThread` because it requires Debugger::DebugThread to be loaded, which it
    # won't when not debugging.
    thread.class.name == 'Debugger::DebugThread' ||
      thread.class.name == 'Debase::DebugThread'
  }
end