Passed
Pull Request — master (#60)
by Juan José
01:43
created

ospd_openvas.wrapper   F

Complexity

Total Complexity 160

Size/Duplication

Total Lines 1167
Duplicated Lines 81.75 %

Importance

Changes 0
Metric Value
eloc 790
dl 954
loc 1167
rs 1.81
c 0
b 0
f 0
wmc 160

36 Methods

Rating   Name   Duplication   Size   Complexity  
A OSPDopenvas.get_modification_time_vt_as_xml_str() 12 12 1
C OSPDopenvas.build_credentials_as_prefs() 73 73 7
A OSPDopenvas.scan_is_stopped() 9 9 1
A OSPDopenvas.get_detection_vt_as_xml_str() 22 22 4
A OSPDopenvas.scheduler() 3 3 1
A OSPDopenvas.get_summary_vt_as_xml_str() 12 12 1
A OSPDopenvas.get_insight_vt_as_xml_str() 12 12 1
B OSPDopenvas.get_refs_vt_as_xml_str() 32 32 8
A OSPDopenvas.get_openvas_timestamp_scan_host() 12 12 3
A OSPDopenvas.get_solution_vt_as_xml_str() 15 15 2
F OSPDopenvas.exec_scan() 154 154 19
A OSPDopenvas.stop_scan() 19 19 3
A OSPDopenvas.get_custom_vt_as_xml_str() 16 16 2
A OSPDopenvas.get_severity_score() 17 17 3
A OSPDopenvas.get_vt_param_type() 6 6 2
A OSPDopenvas.get_impact_vt_as_xml_str() 13 13 1
A OSPDopenvas.check() 22 22 4
B OSPDopenvas.check_param_type() 22 22 8
A OSPDopenvas.get_severities_vt_as_xml_str() 19 19 4
B OSPDopenvas.get_vts_in_groups() 21 21 6
A OSPDopenvas.get_params_vt_as_xml_str() 22 22 3
A OSPDopenvas.update_progress() 21 21 3
A OSPDopenvas.get_dependencies_vt_as_xml_str() 21 21 3
C OSPDopenvas.get_openvas_result() 59 59 8
A OSPDopenvas.get_openvas_status() 10 10 2
F OSPDopenvas.load_vts() 93 93 18
B OSPDopenvas.process_vts() 29 29 7
A OSPDopenvas.get_affected_vt_as_xml_str() 12 12 1
A OSPDopenvas.scan_is_finished() 4 4 1
C OSPDopenvas.check_feed() 30 30 10
A OSPDopenvas.get_creation_time_vt_as_xml_str() 12 12 1
A OpenVasVtsFilter.format_vt_modification_time() 10 10 1
A OSPDopenvas.feed_is_outdated() 21 21 5
A OSPDopenvas.redis_nvticache_init() 8 8 2
A OSPDopenvas.__init__() 26 26 3
B OSPDopenvas.parse_param() 22 22 8

2 Functions

Rating   Name   Duplication   Size   Complexity  
A main() 0 3 1
A _from_bool_to_str() 0 4 2

How to fix   Duplicated Code    Complexity   

Duplicated Code

Duplicate code is one of the most pungent code smells. A rule that is often used is to re-structure code once it is duplicated in three or more places.

Common duplication problems, and corresponding solutions are:

Complexity

 Tip:   Before tackling complexity, make sure that you eliminate any duplication first. This often can reduce the size of classes significantly.

Complex classes like ospd_openvas.wrapper often do a lot of different things. To break such a class down, we need to identify a cohesive component within that class. A common approach to find such a component is to look for fields/methods that share the same prefixes, or suffixes.

Once you have determined the fields that belong together, you can apply the Extract Class refactoring. If the component makes sense as a sub-class, Extract Subclass is also a candidate, and is often faster.

1
# -*- coding: utf-8 -*-
2
# Copyright (C) 2018 Greenbone Networks GmbH
3
#
4
# SPDX-License-Identifier: GPL-2.0-or-later
5
#
6
# This program is free software; you can redistribute it and/or
7
# modify it under the terms of the GNU General Public License
8
# as published by the Free Software Foundation; either version 2
9
# of the License, or (at your option) any later version.
10
#
11
# This program is distributed in the hope that it will be useful,
12
# but WITHOUT ANY WARRANTY; without even the implied warranty of
13
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
14
# GNU General Public License for more details.
15
#
16
# You should have received a copy of the GNU General Public License
17
# along with this program; if not, write to the Free Software
18
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
19
20
""" Setup for the OSP OpenVAS Server. """
21
22
import subprocess
23
import time
24
import signal
25
import uuid
26
from lxml.etree import tostring, SubElement, Element
27
from datetime import datetime
28
import psutil
29
from os import path
30
31
from ospd.ospd import OSPDaemon, logger
32
from ospd.misc import main as daemon_main
33
from ospd.misc import target_str_to_list
34
from ospd.cvss import CVSS
35
from ospd.vtfilter import VtsFilter
36
from ospd_openvas import __version__
37
from ospd_openvas.errors import OSPDOpenvasError
38
39
from ospd_openvas.nvticache import NVTICache
40
from ospd_openvas.db import OpenvasDB
41
42
OSPD_DESC = """
43
This scanner runs 'OpenVAS Scanner' to scan the target hosts.
44
45
OpenVAS (Open Vulnerability Assessment System) is a powerful scanner
46
for vulnerabilities in IT infrastrucutres. The capabilities include
47
unauthzenticated scanning as well as authneticated scanning for
48
various types of systems and services.
49
50
For more details about OpenVAS see the OpenVAS homepage:
51
http://www.openvas.org/
52
53
The current version of ospd-openvas is a simple frame, which sends
54
the server parameters to the Greenbone Vulnerability Manager (GVM) and checks
55
the existence of OpenVAS Scanner binary. But it can not run scans yet.
56
"""
57
58
OSPD_PARAMS = {
59
    'auto_enable_dependencies': {
60
        'type': 'boolean',
61
        'name': 'auto_enable_dependencies',
62
        'default': 1,
63
        'mandatory': 1,
64
        'description': 'Automatically enable the plugins that are depended on',
65
    },
66
    'cgi_path': {
67
        'type': 'string',
68
        'name': 'cgi_path',
69
        'default': '/cgi-bin:/scripts',
70
        'mandatory': 1,
71
        'description': 'Look for default CGIs in /cgi-bin and /scripts',
72
    },
73
    'checks_read_timeout': {
74
        'type': 'integer',
75
        'name': 'checks_read_timeout',
76
        'default': 5,
77
        'mandatory': 1,
78
        'description': ('Number  of seconds that the security checks will ' +
79
                        'wait for when doing a recv()'),
80
    },
81
    'drop_privileges': {
82
        'type': 'boolean',
83
        'name': 'drop_privileges',
84
        'default': 0,
85
        'mandatory': 1,
86
        'description': '',
87
    },
88
    'network_scan': {
89
        'type': 'boolean',
90
        'name': 'network_scan',
91
        'default': 0,
92
        'mandatory': 1,
93
        'description': '',
94
    },
95
    'non_simult_ports': {
96
        'type': 'string',
97
        'name': 'non_simult_ports',
98
        'default': '139, 445, 3389, Services/irc',
99
        'mandatory': 1,
100
        'description': ('Prevent to make two connections on the same given ' +
101
                        'ports at the same time.'),
102
    },
103
    'open_sock_max_attempts': {
104
        'type': 'integer',
105
        'name': 'open_sock_max_attempts',
106
        'default': 5,
107
        'mandatory': 0,
108
        'description': ('Number of unsuccessful retries to open the socket ' +
109
                        'before to set the port as closed.'),
110
    },
111
    'timeout_retry': {
112
        'type': 'integer',
113
        'name': 'timeout_retry',
114
        'default': 5,
115
        'mandatory': 0,
116
        'description': ('Number of retries when a socket connection attempt ' +
117
                        'timesout.'),
118
    },
119
    'optimize_test': {
120
        'type': 'integer',
121
        'name': 'optimize_test',
122
        'default': 5,
123
        'mandatory': 0,
124
        'description': ('By default, openvassd does not trust the remote ' +
125
                        'host banners.'),
126
    },
127
    'plugins_timeout': {
128
        'type': 'integer',
129
        'name': 'plugins_timeout',
130
        'default': 5,
131
        'mandatory': 0,
132
        'description': 'This is the maximum lifetime, in seconds of a plugin.',
133
    },
134
    'report_host_details': {
135
        'type': 'boolean',
136
        'name': 'report_host_details',
137
        'default': 1,
138
        'mandatory': 1,
139
        'description': '',
140
    },
141
    'safe_checks': {
142
        'type': 'boolean',
143
        'name': 'safe_checks',
144
        'default': 1,
145
        'mandatory': 1,
146
        'description': ('Disable the plugins with potential to crash ' +
147
                        'the remote services'),
148
    },
149
    'scanner_plugins_timeout': {
150
        'type': 'integer',
151
        'name': 'scanner_plugins_timeout',
152
        'default': 36000,
153
        'mandatory': 1,
154
        'description': 'Like plugins_timeout, but for ACT_SCANNER plugins.',
155
    },
156
    'time_between_request': {
157
        'type': 'integer',
158
        'name': 'time_between_request',
159
        'default': 0,
160
        'mandatory': 0,
161
        'description': ('Allow to set a wait time between two actions ' +
162
                        '(open, send, close).'),
163
    },
164
    'unscanned_closed': {
165
        'type': 'boolean',
166
        'name': 'unscanned_closed',
167
        'default': 1,
168
        'mandatory': 1,
169
        'description': '',
170
    },
171
    'unscanned_closed_udp': {
172
        'type': 'boolean',
173
        'name': 'unscanned_closed_udp',
174
        'default': 1,
175
        'mandatory': 1,
176
        'description': '',
177
    },
178
    'use_mac_addr': {
179
        'type': 'boolean',
180
        'name': 'use_mac_addr',
181
        'default': 0,
182
        'mandatory': 0,
183
        'description': 'To test the local network. ' +
184
                       'Hosts will be referred to by their MAC address.',
185
    },
186
    'vhosts': {
187
        'type': 'string',
188
        'name': 'vhosts',
189
        'default': '',
190
        'mandatory': 0,
191
        'description': '',
192
    },
193
    'vhosts_ip': {
194
        'type': 'string',
195
        'name': 'vhosts_ip',
196
        'default': '',
197
        'mandatory': 0,
198
        'description': '',
199
    },
200
}
201
202
203
def _from_bool_to_str(value):
204
    """ The OpenVAS scanner use yes and no as boolean values, whereas ospd
205
    uses 1 and 0."""
206
    return 'yes' if value == 1 else 'no'
207
208 View Code Duplication
class OpenVasVtsFilter(VtsFilter):
0 ignored issues
show
Duplication introduced by
This code seems to be duplicated in your project.
Loading history...
209
    """ Methods to overwrite the ones in the original class.
210
    Each method formats the value to be compatible with the filter
211
    """
212
    def format_vt_modification_time(self, value):
213
        """ Convert the datetime value in an 19 character string
214
        representing YearMonthDateHourMinuteSecond.
215
        e.g. 20190319122532
216
        """
217
218
        date = value[7:26].replace(" ", "")
219
        date = date.replace("-", "")
220
        date = date.replace(":","")
221
        return date
222
223 View Code Duplication
class OSPDopenvas(OSPDaemon):
0 ignored issues
show
Duplication introduced by
This code seems to be duplicated in your project.
Loading history...
224
225
    """ Class for ospd-openvas daemon. """
226
227
    def __init__(self, certfile, keyfile, cafile):
228
        """ Initializes the ospd-openvas daemon's internal data. """
229
230
        super().__init__(certfile=certfile, keyfile=keyfile, cafile=cafile,
231
                         customvtfilter=OpenVasVtsFilter())
232
        self.server_version = __version__
233
        self.scanner_info['name'] = 'openvassd'
234
        self.scanner_info['version'] = ''  # achieved during self.check()
235
        self.scanner_info['description'] = OSPD_DESC
236
        for name, param in OSPD_PARAMS.items():
237
            self.add_scanner_param(name, param)
238
239
        self.scan_only_params = dict()
240
        self.main_kbindex = None
241
        self.openvas_db = OpenvasDB()
242
        self.nvti = NVTICache(self.openvas_db)
243
244
        self.openvas_db.db_init()
245
246
        self.pending_feed = None
247
        ctx = self.openvas_db.db_find(self.nvti.NVTICACHE_STR)
248
        if not ctx:
249
            self.redis_nvticache_init()
250
            ctx = self.openvas_db.db_find(self.nvti.NVTICACHE_STR)
251
        self.openvas_db.set_redisctx(ctx)
252
        self.load_vts()
253
254
    def parse_param(self):
255
        """ Set OSPD_PARAMS with the params taken from the openvas_scanner. """
256
        global OSPD_PARAMS
257
        bool_dict = {'no': 0, 'yes': 1}
258
259
        result = subprocess.check_output(['openvassd', '-s'],
260
                                         stderr=subprocess.STDOUT)
261
        result = result.decode('ascii')
262
        param_list = dict()
263
        for conf in result.split('\n'):
264
            elem = conf.split('=')
265
            if len(elem) == 2:
266
                value = str.strip(elem[1])
267
                if str.strip(elem[1]) in bool_dict:
268
                    value = bool_dict[value]
269
                param_list[str.strip(elem[0])] = value
270
        for elem in OSPD_PARAMS:
271
            if elem in param_list:
272
                OSPD_PARAMS[elem]['default'] = param_list[elem]
273
        for elem in param_list:
274
            if elem not in OSPD_PARAMS:
275
                self.scan_only_params[elem] = param_list[elem]
276
277
    def redis_nvticache_init(self):
278
        """ Loads NVT's metadata into Redis DB. """
279
        try:
280
            logger.debug('Loading NVTs in Redis DB')
281
            subprocess.check_call(['openvassd', '-C'])
282
        except subprocess.CalledProcessError as err:
283
            logger.error('OpenVAS Scanner failed to load NVTs.')
284
            raise err
285
286
    def feed_is_outdated(self):
287
        """ Compare the current feed with the one in the disk.
288
289
        Return:
290
            False if there is no new feed. True if the feed version in disk
291
            is newer than the feed in redis cache.
292
        """
293
        plugins_folder = self.scan_only_params.get('plugins_folder')
294
        if not plugins_folder:
295
            raise OSPDOpenvasError("Error: Path to plugins folder not found.")
296
        feed_info_file = path.join(plugins_folder, 'plugin_feed_info.inc')
297
        fcontent = open(feed_info_file)
298
        for line in fcontent:
299
            if "PLUGIN_SET" in line:
300
                date = line.split(' = ')[1]
301
                date = date.replace(';', '')
302
                date = date.replace('"', '')
303
        current_feed =  self.nvti.get_feed_version()
304
        if int(current_feed) < int(date):
0 ignored issues
show
introduced by
The variable date does not seem to be defined for all execution paths.
Loading history...
305
            return True
306
        return False
307
308
    def check_feed(self):
309
        """ Check if there is a feed update. Wait until all the running
310
        scans finished. Set a flag to anounce there is a pending feed update,
311
        which avoid to start a new scan.
312
        """
313
314
        # Check if the nvticache in redis is outdated
315
        if self.feed_is_outdated():
316
            self.redis_nvticache_init()
317
318
        _running_scan = False
319
        for scan_id in self.scan_processes:
320
            if self.scan_processes[scan_id].is_alive():
321
                _running_scan = True
322
323
        # Check if the NVT dict is outdated
324
        if self.pending_feed:
325
            _pending_feed = True
326
        else:
327
            _pending_feed = self.get_vts_version() != self.nvti.get_feed_version()
328
329
        if _running_scan and _pending_feed:
330
            if not self.pending_feed:
331
                self.pending_feed = True
332
                logger.debug(
333
                    'There is a running scan. Therefore the feed '
334
                    'update will be performed later.')
335
        elif not _running_scan and _pending_feed:
336
            self.vts = dict()
337
            self.load_vts()
338
339
    def scheduler(self):
340
        """This method is called periodically to run tasks."""
341
        self.check_feed()
342
343
    def load_vts(self):
344
        """ Load the NVT's metadata into the vts
345
        global  dictionary. """
346
        logger.debug('Loading vts in memory.')
347
        oids = dict(self.nvti.get_oids())
348
        for filename, vt_id in oids.items():
349
            _vt_params = self.nvti.get_nvt_params(vt_id)
350
            _vt_refs = self.nvti.get_nvt_refs(vt_id)
351
            _custom = self.nvti.get_nvt_metadata(vt_id)
352
            _name = _custom.pop('name')
353
            _vt_creation_time = _custom.pop('creation_date')
354
            _vt_modification_time = _custom.pop('last_modification')
355
356
            _summary = None
357
            _impact = None
358
            _affected = None
359
            _insight = None
360
            _solution = None
361
            _solution_t = None
362
            _vuldetect = None
363
            _qod_t = None
364
            _qod_v = None
365
366
            if 'summary' in _custom:
367
                _summary = _custom.pop('summary')
368
            if 'impact' in _custom:
369
                _impact = _custom.pop('impact')
370
            if 'affected' in _custom:
371
                _affected = _custom.pop('affected')
372
            if 'insight' in _custom :
373
                _insight = _custom.pop('insight')
374
            if 'solution' in _custom:
375
                _solution = _custom.pop('solution')
376
                if 'solution_type' in _custom:
377
                    _solution_t = _custom.pop('solution_type')
378
379
            if 'vuldetect' in _custom:
380
                _vuldetect  = _custom.pop('vuldetect')
381
            if 'qod_type' in _custom:
382
                _qod_t  = _custom.pop('qod_type')
383
            elif 'qod' in _custom:
384
                _qod_v  = _custom.pop('qod')
385
386
            _severity = dict()
387
            if 'severity_base_vector' in _custom:
388
                _severity_vector = _custom.pop('severity_base_vector')
389
            else:
390
                _severity_vector = _custom.pop('cvss_base_vector')
391
            _severity['severity_base_vector'] = _severity_vector
392
            if 'severity_type' in _custom:
393
                _severity_type = custom.pop('severity_type')
0 ignored issues
show
Comprehensibility Best Practice introduced by
The variable custom does not seem to be defined.
Loading history...
394
            else:
395
                _severity_type = 'cvss_base_v2'
396
            _severity['severity_type'] = _severity_type
397
            if 'severity_origin' in _custom:
398
                _severity['severity_origin'] = _custom.pop('severity_origin')
399
400
            _vt_dependencies = list()
401
            if 'dependencies' in _custom:
402
                _deps = _custom.pop('dependencies')
403
                _deps_list = _deps.split(', ')
404
                for dep in _deps_list:
405
                    _vt_dependencies.append(oids.get('filename:' + dep))
406
407
            ret = self.add_vt(
408
                vt_id,
409
                name=_name,
410
                vt_params=_vt_params,
411
                vt_refs=_vt_refs,
412
                custom=_custom,
413
                vt_creation_time=_vt_creation_time,
414
                vt_modification_time=_vt_modification_time,
415
                vt_dependencies=_vt_dependencies,
416
                summary=_summary,
417
                impact=_impact,
418
                affected=_affected,
419
                insight=_insight,
420
                solution=_solution,
421
                solution_t=_solution_t,
422
                detection=_vuldetect,
423
                qod_t=_qod_t,
424
                qod_v=_qod_v,
425
                severities=_severity
426
            )
427
            if ret == -1:
428
                logger.info("Dupplicated VT with OID: {0}".format(vt_id))
429
            if ret == -2:
430
                logger.info("{0}: Invalid OID.".format(vt_id))
431
432
        _feed_version = self.nvti.get_feed_version()
433
        self.set_vts_version(vts_version=_feed_version)
434
        self.pending_feed = False
435
        logger.debug('Finish loading up vts.')
436
437
    @staticmethod
438
    def get_custom_vt_as_xml_str(vt_id, custom):
439
        """ Return an xml element with custom metadata formatted as string.
440
        Arguments:
441
            vt_id (str): VT OID. Only used for logging in error case.
442
            custom (dict): Dictionary with the custom metadata.
443
        Return:
444
            string: xml element as string.
445
        """
446
447
        _custom = Element('custom')
448
        for key, val in custom.items():
449
            xml_key = SubElement(_custom, key)
450
            xml_key.text = val
451
452
        return tostring(_custom).decode('utf-8')
453
454
    @staticmethod
455
    def get_severities_vt_as_xml_str(vt_id, severities):
456
        """ Return an xml element with severities as string.
457
        Arguments:
458
            vt_id (str): VT OID. Only used for logging in error case.
459
            severities (dict): Dictionary with the severities.
460
        Return:
461
            string: xml element as string.
462
        """
463
        _severities = Element('severities')
464
        _severity = SubElement(_severities, 'severity')
465
        if 'severity_base_vector' in severities:
466
            _severity.text = severities.get('severity_base_vector')
467
        if 'severity_origin' in severities:
468
            _severity.set('origin', severities.get('severity_origin'))
469
        if 'severity_type' in severities:
470
            _severity.set('type', severities.get('severity_type'))
471
472
        return tostring(_severities).decode('utf-8')
473
474
    @staticmethod
475
    def get_params_vt_as_xml_str(vt_id, vt_params):
476
        """ Return an xml element with params formatted as string.
477
        Arguments:
478
            vt_id (str): VT OID. Only used for logging in error case.
479
            vt_params (dict): Dictionary with the VT parameters.
480
        Return:
481
            string: xml element as string.
482
        """
483
        vt_params_xml = Element('vt_params')
484
        for pref_name, prefs in vt_params.items():
485
            vt_param = Element('vt_param')
486
            vt_param.set('type', prefs['type'])
487
            vt_param.set('id', prefs['id'])
488
            xml_name = SubElement(vt_param, 'name')
489
            xml_name.text = prefs['name']
490
            if prefs['default']:
491
                xml_def = SubElement(vt_param, 'default')
492
                xml_def.text = prefs['default']
493
            vt_params_xml.append(vt_param)
494
495
        return tostring(vt_params_xml).decode('utf-8')
496
497
    @staticmethod
498
    def get_refs_vt_as_xml_str(vt_id, vt_refs):
499
        """ Return an xml element with references formatted as string.
500
        Arguments:
501
            vt_id (str): VT OID. Only used for logging in error case.
502
            vt_refs (dict): Dictionary with the VT references.
503
        Return:
504
            string: xml element as string.
505
        """
506
        vt_refs_xml = Element('refs')
507
        for ref_type, ref_values in vt_refs.items():
508
            for value in ref_values:
509
                vt_ref = Element('ref')
510
                if ref_type == "xref" and value:
511
                    for xref in value.split(', '):
512
                        try:
513
                            _type, _id = xref.split(':', 1)
514
                        except ValueError:
515
                            logger.error(
516
                                'Not possible to parse xref %s for vt %s' % (
517
                                    xref, vt_id))
518
                            continue
519
                        vt_ref.set('type', _type.lower())
520
                        vt_ref.set('id', _id)
521
                elif value:
522
                    vt_ref.set('type', ref_type.lower())
523
                    vt_ref.set('id', value)
524
                else:
525
                    continue
526
                vt_refs_xml.append(vt_ref)
527
528
        return tostring(vt_refs_xml).decode('utf-8')
529
530
    @staticmethod
531
    def get_dependencies_vt_as_xml_str(vt_id, dep_list):
532
        """ Return  an xml element with dependencies as string.
533
        Arguments:
534
            vt_id (str): VT OID. Only used for logging in error case.
535
            dep_list (List): List with the VT dependencies.
536
        Return:
537
            string: xml element as string.
538
        """
539
        vt_deps_xml = Element('dependencies')
540
        for dep in dep_list:
541
            _vt_dep = Element('dependency')
542
            try:
543
                _vt_dep.set('vt_id', dep)
544
            except TypeError:
545
                logger.error('Not possible to add dependency %s for vt %s' % (
546
                    dep, vt_id))
547
                continue
548
            vt_deps_xml.append(_vt_dep)
549
550
        return tostring(vt_deps_xml).decode('utf-8')
551
552
    @staticmethod
553
    def get_creation_time_vt_as_xml_str(vt_id, creation_time):
554
        """ Return creation time as string.
555
        Arguments:
556
            vt_id (str): VT OID. Only used for logging in error case.
557
            creation_time (str): String with the VT creation time.
558
        Return:
559
            string: xml element as string.
560
        """
561
        _time = Element('creation_time')
562
        _time.text = creation_time
563
        return tostring(_time).decode('utf-8')
564
565
    @staticmethod
566
    def get_modification_time_vt_as_xml_str(vt_id, modification_time):
567
        """ Return modification time as string.
568
        Arguments:
569
            vt_id (str): VT OID. Only used for logging in error case.
570
            modification_time (str): String with the VT modification time.
571
        Return:
572
            string: xml element as string.
573
        """
574
        _time = Element('modification_time')
575
        _time.text = modification_time
576
        return tostring(_time).decode('utf-8')
577
578
    @staticmethod
579
    def get_summary_vt_as_xml_str(vt_id, summary):
580
        """ Return summary as string.
581
        Arguments:
582
            vt_id (str): VT OID. Only used for logging in error case.
583
            summary (str): String with a VT summary.
584
        Return:
585
            string: xml element as string.
586
        """
587
        _summary = Element('summary')
588
        _summary.text = summary
589
        return tostring(_summary).decode('utf-8')
590
591
    @staticmethod
592
    def get_impact_vt_as_xml_str(vt_id, impact):
593
        """ Return impact as string.
594
595
        Arguments:
596
            vt_id (str): VT OID. Only used for logging in error case.
597
            impact (str): String which explain the vulneravility impact.
598
        Return:
599
            string: xml element as string.
600
        """
601
        _impact = Element('impact')
602
        _impact.text = impact
603
        return tostring(_impact).decode('utf-8')
604
605
    @staticmethod
606
    def get_affected_vt_as_xml_str(vt_id, affected):
607
        """ Return affected as string.
608
        Arguments:
609
            vt_id (str): VT OID. Only used for logging in error case.
610
            affected (str): String which explain what is affected.
611
        Return:
612
            string: xml element as string.
613
        """
614
        _affected = Element('affected')
615
        _affected.text = affected
616
        return tostring(_affected).decode('utf-8')
617
618
    @staticmethod
619
    def get_insight_vt_as_xml_str(vt_id, insight):
620
        """ Return insight as string.
621
        Arguments:
622
            vt_id (str): VT OID. Only used for logging in error case.
623
            insight (str): String giving an insight of the vulnerability.
624
        Return:
625
            string: xml element as string.
626
        """
627
        _insight = Element('insight')
628
        _insight.text = insight
629
        return tostring(_insight).decode('utf-8')
630
631
    @staticmethod
632
    def get_solution_vt_as_xml_str(vt_id, solution, solution_type=None):
633
        """ Return solution as string.
634
        Arguments:
635
            vt_id (str): VT OID. Only used for logging in error case.
636
            solution (str): String giving a possible solution.
637
            solution_type (str): A solution type
638
        Return:
639
            string: xml element as string.
640
        """
641
        _solution = Element('solution')
642
        _solution.text = solution
643
        if solution_type:
644
            _solution.set('type', solution_type)
645
        return tostring(_solution).decode('utf-8')
646
647
    @staticmethod
648
    def get_detection_vt_as_xml_str(vt_id, vuldetect=None,
649
                                    qod_type=None, qod=None):
650
        """ Return detection as string.
651
        Arguments:
652
            vt_id (str): VT OID. Only used for logging in error case.
653
            vuldetect (str, opt): String which explain how the vulnerability
654
                was detected.
655
            qod_type (str, opt): qod type.
656
            qod (str, opt): qod value.
657
        Return:
658
            string: xml element as string.
659
        """
660
        _detection = Element('detection')
661
        if vuldetect:
662
            _detection.text = vuldetect
663
        if qod_type:
664
            _detection.set('qod_type', qod_type)
665
        elif qod:
666
            _detection.set('qod', qod)
667
668
        return tostring(_detection).decode('utf-8')
669
670
    def check(self):
671
        """ Checks that openvassd command line tool is found and
672
        is executable. """
673
        try:
674
            result = subprocess.check_output(['openvassd', '-V'],
675
                                             stderr=subprocess.STDOUT)
676
            result = result.decode('ascii')
677
        except OSError:
678
            # The command is not available
679
            return False
680
681
        if result is None:
682
            return False
683
684
        version = result.split('\n')
685
        if version[0].find('OpenVAS') < 0:
686
            return False
687
688
        self.parse_param()
689
        self.scanner_info['version'] = version[0]
690
691
        return True
692
693
    def update_progress(self, scan_id, target, msg):
694
        """ Calculate percentage and update the scan status of a target
695
        for the progress bar.
696
        Arguments:
697
            scan_id (uuid): Scan ID to identify the current scan process.
698
            target (str): Target to be updated with the calculated
699
                          scan progress.
700
            msg (str): String with launched and total plugins.
701
        """
702
        host_progress_dict = dict()
703
        try:
704
            launched, total = msg.split('/')
705
        except ValueError:
706
            return
707
        if float(total) == 0:
708
            return
709
        host_prog = (float(launched) / float(total)) * 100
710
        host_progress_dict[target] = host_prog
711
        total_host = len(target_str_to_list(target))
712
        target_progress = sum(host_progress_dict.values()) / total_host
713
        self.set_scan_target_progress(scan_id, target, target_progress)
714
715
    def get_openvas_status(self, scan_id, target):
716
        """ Get all status entries from redis kb.
717
        Arguments:
718
            scan_id (uuid): Scan ID to identify the current scan.
719
            target (str): Target progress to be updated.
720
        """
721
        res = self.openvas_db.get_status()
722
        while res:
723
            self.update_progress(scan_id, target, res)
724
            res = self.openvas_db.get_status()
725
726
    def get_severity_score(self, oid):
727
        """ Return the severity score for the given oid.
728
        Arguments:
729
            oid (str): VT OID from which to get the severity vector
730
        Returns:
731
            The calculated cvss base value. None if there is no severity
732
            vector or severity type is not cvss base version 2.
733
        """
734
        severity_type = (
735
            self.vts[oid]['severities'].get('severity_type'))
736
        severity_vector = (
737
            self.vts[oid]['severities'].get('severity_base_vector'))
738
739
        if severity_type == "cvss_base_v2" and severity_vector:
740
            return CVSS.cvss_base_v2_value(severity_vector)
741
742
        return None
743
744
    def get_openvas_result(self, scan_id):
745
        """ Get all result entries from redis kb. """
746
        res = self.openvas_db.get_result()
747
        while res:
748
            msg = res.split('|||')
749
            host_aux = self.openvas_db.get_single_item('internal/ip')
750
            roid = msg[3]
751
752
            rqod = ''
753
            if self.vts[roid].get('qod_type'):
754
                qod_t = self.vts[roid].get('qod_type')
755
                rqod = self.nvti.QoD_TYPES[qod_t]
756
            elif self.vts[roid].get('qod'):
757
                rqod = self.vts[roid].get('qod')
758
759
            rname = self.vts[roid].get('name')
760
761
            if msg[0] == 'ERRMSG':
762
                self.add_scan_error(
763
                    scan_id,
764
                    host=host_aux,
765
                    name=rname,
766
                    value=msg[4],
767
                    port=msg[2],
768
                )
769
770
            if msg[0] == 'LOG':
771
                self.add_scan_log(
772
                    scan_id,
773
                    host=host_aux,
774
                    name=rname,
775
                    value=msg[4],
776
                    port=msg[2],
777
                    qod=rqod,
778
                    test_id=roid,
779
                )
780
781
            if msg[0] == 'HOST_DETAIL':
782
                self.add_scan_host_detail(
783
                    scan_id,
784
                    host=host_aux,
785
                    name=rname,
786
                    value=msg[4],
787
                )
788
789
            if msg[0] == 'ALARM':
790
                rseverity = self.get_severity_score(roid)
791
                self.add_scan_alarm(
792
                    scan_id,
793
                    host=host_aux,
794
                    name=rname,
795
                    value=msg[4],
796
                    port=msg[2],
797
                    test_id=roid,
798
                    severity=rseverity,
799
                    qod=rqod,
800
                )
801
802
            res = self.openvas_db.get_result()
803
804
    def get_openvas_timestamp_scan_host(self, scan_id, target):
805
        """ Get start and end timestamp of a host scan from redis kb. """
806
        timestamp = self.openvas_db.get_host_scan_scan_end_time()
807
        if timestamp:
808
            self.add_scan_log(scan_id, host=target, name='HOST_END',
809
                              value=timestamp)
810
            return
811
        timestamp = self.openvas_db.get_host_scan_scan_start_time()
812
        if timestamp:
813
            self.add_scan_log(scan_id, host=target, name='HOST_START',
814
                              value=timestamp)
815
            return
816
817
    def scan_is_finished(self, scan_id):
818
        """ Check if the scan has finished. """
819
        status = self.openvas_db.get_single_item('internal/%s' % scan_id)
820
        return status == 'finished'
821
822
    def scan_is_stopped(self, scan_id):
823
        """ Check if the parent process has received the stop_scan order.
824
        @in scan_id: ID to identify the scan to be stopped.
825
        @return 1 if yes, None in other case.
826
        """
827
        ctx = self.openvas_db.kb_connect(dbnum=self.main_kbindex)
828
        self.openvas_db.set_redisctx(ctx)
829
        status = self.openvas_db.get_single_item('internal/%s' % scan_id)
830
        return status == 'stop_all'
831
832
    def stop_scan(self, global_scan_id):
833
        """ Set a key in redis to indicate the wrapper is stopped.
834
        It is done through redis because it is a new multiprocess
835
        instance and it is not possible to reach the variables
836
        of the grandchild process. Send SIGUSR2 to openvas to stop
837
        each running scan."""
838
        ctx = self.openvas_db.kb_connect()
839
        for current_kbi in range(0, self.openvas_db.max_dbindex):
840
            self.openvas_db.select_kb(ctx, str(current_kbi), set_global=True)
841
            scan_id = self.openvas_db.get_single_item(
842
                'internal/%s/globalscanid' % global_scan_id)
843
            if scan_id:
844
                self.openvas_db.set_single_item('internal/%s' % scan_id,
845
                                           ['stop_all', ])
846
                ovas_pid = self.openvas_db.get_single_item('internal/ovas_pid')
847
                parent = psutil.Process(int(ovas_pid))
848
                self.openvas_db.release_db(current_kbi)
849
                parent.send_signal(signal.SIGUSR2)
850
                logger.debug('Stopping process: {0}'.format(parent))
851
852
    def get_vts_in_groups(self, filters):
853
        """ Return a list of vts which match with the given filter.
854
855
        @input filters A list of filters. Each filter has key, operator and
856
                       a value. They are separated by a space.
857
                       Supported keys: family
858
        @return Return a list of vts which match with the given filter.
859
        """
860
        vts_list = list()
861
        families = dict()
862
        for oid in self.vts:
863
            family = self.vts[oid]['custom'].get('family')
864
            if family not in families:
865
                families[family] = list()
866
            families[family].append(oid)
867
868
        for elem in filters:
869
            key, value = elem.split('=')
870
            if key == 'family' and value in families:
871
                vts_list.extend(families[value])
872
        return vts_list
873
874
    def get_vt_param_type(self, vtid, vt_param_id):
875
        """ Return the type of the vt parameter from the vts dictionary. """
876
        vt_params_list = self.vts[vtid].get("vt_params")
877
        if vt_params_list.get(vt_param_id):
878
            return vt_params_list[vt_param_id]["type"]
879
        return False
880
881
    @staticmethod
882
    def check_param_type(vt_param_value, param_type):
883
        """ Check if the value of a vt parameter matches with
884
        the type founded.
885
        """
886
        if (param_type in ['entry',
887
                           'file',
888
                           'password',
889
                           'radio',
890
                           'sshlogin', ] and isinstance(vt_param_value, str)):
891
            return None
892
        elif (param_type == 'checkbox' and
893
              (vt_param_value == 'yes' or vt_param_value == 'no')):
894
            return None
895
        elif param_type == 'integer':
896
            try:
897
                int(vt_param_value)
898
            except ValueError:
899
                return 1
900
            return None
901
902
        return 1
903
904
    def process_vts(self, vts):
905
        """ Add single VTs and their parameters. """
906
        vts_list = []
907
        vts_params = []
908
        vtgroups = vts.pop('vt_groups')
909
910
        if vtgroups:
911
            vts_list = self.get_vts_in_groups(vtgroups)
912
913
        for vtid, vt_params in vts.items():
914
            vts_list.append(vtid)
915
            for vt_param_id, vt_param_value in vt_params.items():
916
                param_type = self.get_vt_param_type(vtid, vt_param_id)
917
                if not param_type:
918
                    logger.debug('The vt parameter %s for %s could not be loaded.',
919
                                 vt_param_id, vtid)
920
                    continue
921
                if vt_param_id == 'timeout':
922
                    type_aux = 'integer'
923
                else:
924
                    type_aux = param_type
925
                if self.check_param_type(vt_param_value, type_aux):
926
                    logger.debug('Expected {} type for parameter value {}'
927
                                 .format(type_aux, str(vt_param_value)))
928
                param = ["{0}:{1}:{2}".format(vtid, param_type,
929
                                               vt_param_id),
930
                         str(vt_param_value)]
931
                vts_params.append(param)
932
        return vts_list, vts_params
933
934
    @staticmethod
935
    def build_credentials_as_prefs(credentials):
936
        """ Parse the credential dictionary.
937
        @param credentials: Dictionary with the credentials.
938
939
        @return A list with the credentials in string format to be
940
                added to the redis KB.
941
        """
942
        cred_prefs_list = []
943
        for credential in credentials.items():
944
            service = credential[0]
945
            cred_params = credentials.get(service)
946
            cred_type = cred_params.get('type', '')
947
            username = cred_params.get('username', '')
948
            password = cred_params.get('password', '')
949
950
            if service == 'ssh':
951
                port = cred_params.get('port', '')
952
                cred_prefs_list.append('auth_port_ssh|||' +
953
                                       '{0}'.format(port))
954
                cred_prefs_list.append('SSH Authorization[entry]:SSH login ' +
955
                                       'name:|||{0}'.format(username))
956
                if cred_type == 'up':
957
                    cred_prefs_list.append('SSH Authorization[password]:' +
958
                                           'SSH password (unsafe!):|||' +
959
                                           '{0}'.format(password))
960
                else:
961
                    private = cred_params.get('private', '')
962
                    cred_prefs_list.append('SSH Authorization[password]:' +
963
                                           'SSH key passphrase:|||' +
964
                                           '{0}'.format(password))
965
                    cred_prefs_list.append('SSH Authorization[file]:' +
966
                                           'SSH private key:|||' +
967
                                           '{0}'.format(private))
968
            if service == 'smb':
969
                cred_prefs_list.append('SMB Authorization[entry]:SMB login:' +
970
                                       '|||{0}'.format(username))
971
                cred_prefs_list.append('SMB Authorization[password]:' +
972
                                       'SMB password :|||' +
973
                                       '{0}'.format(password))
974
            if service == 'esxi':
975
                cred_prefs_list.append('ESXi Authorization[entry]:ESXi login ' +
976
                                       'name:|||{0}'.format(username))
977
                cred_prefs_list.append('ESXi Authorization[password]:' +
978
                                       'ESXi login password:|||' +
979
                                       '{0}'.format(password))
980
981
            if service == 'snmp':
982
                community = cred_params.get('community', '')
983
                auth_algorithm = cred_params.get('auth_algorithm', '')
984
                privacy_password = cred_params.get('privacy_password', '')
985
                privacy_algorithm = cred_params.get('privacy_algorithm', '')
986
987
                cred_prefs_list.append('SNMP Authorization[password]:' +
988
                                       'SNMP Community:' +
989
                                       '{0}'.format(community))
990
                cred_prefs_list.append('SNMP Authorization[entry]:' +
991
                                       'SNMPv3 Username:' +
992
                                       '{0}'.format(username))
993
                cred_prefs_list.append('SNMP Authorization[password]:' +
994
                                       'SNMPv3 Password:' +
995
                                       '{0}'.format(password))
996
                cred_prefs_list.append('SNMP Authorization[radio]:' +
997
                                       'SNMPv3 Authentication Algorithm:' +
998
                                       '{0}'.format(auth_algorithm))
999
                cred_prefs_list.append('SNMP Authorization[password]:' +
1000
                                       'SNMPv3 Privacy Password:' +
1001
                                       '{0}'.format(privacy_password))
1002
                cred_prefs_list.append('SNMP Authorization[radio]:' +
1003
                                       'SNMPv3 Privacy Algorithm:' +
1004
                                       '{0}'.format(privacy_algorithm))
1005
1006
        return cred_prefs_list
1007
1008
    def exec_scan(self, scan_id, target):
1009
        """ Starts the OpenVAS scanner for scan_id scan. """
1010
        if self.pending_feed:
1011
            logger.info(
1012
                '%s: There is a pending feed update. '
1013
                'The scan can not be started.' % scan_id)
1014
            self.add_scan_error(
1015
                scan_id, name='', host=target,
1016
                value=('It was not possible to start the scan,'
1017
                'because a pending feed update. Please try later'))
1018
            return 2
1019
1020
        ports = self.get_scan_ports(scan_id, target)
1021
        if not ports:
1022
            self.add_scan_error(scan_id, name='', host=target,
1023
                                value='No port list defined.')
1024
            return 2
1025
1026
        # Get scan options
1027
        options = self.get_scan_options(scan_id)
1028
        prefs_val = []
1029
        ctx = self.openvas_db.kb_new()
1030
        self.openvas_db.set_redisctx(ctx)
1031
        self.main_kbindex = self.openvas_db.db_index
1032
1033
        # To avoid interference between scan process during a parallel scanning
1034
        # new uuid is used internally for each scan.
1035
        openvas_scan_id = str(uuid.uuid4())
1036
        self.openvas_db.add_single_item(
1037
            'internal/%s' % openvas_scan_id, ['new'])
1038
        self.openvas_db.add_single_item(
1039
            'internal/%s/globalscanid' % scan_id, [openvas_scan_id])
1040
1041
        # Set scan preferences
1042
        for key, value in options.items():
1043
            item_type = ''
1044
            if key in OSPD_PARAMS:
1045
                item_type = OSPD_PARAMS[key].get('type')
1046
            if item_type == 'boolean':
1047
                val =  _from_bool_to_str(value)
1048
            else:
1049
                val = str(value)
1050
            prefs_val.append(key + "|||" + val)
1051
        self.openvas_db.add_single_item(
1052
            'internal/%s/scanprefs' % openvas_scan_id, prefs_val)
1053
1054
        # Store main_kbindex as global preference
1055
        ov_maindbid = ('ov_maindbid|||%d' % self.main_kbindex)
1056
        self.openvas_db.add_single_item(
1057
            'internal/%s/scanprefs' % openvas_scan_id, [ov_maindbid])
1058
1059
        # Set target
1060
        target_aux = ('TARGET|||%s' % target)
1061
        self.openvas_db.add_single_item(
1062
            'internal/%s/scanprefs' % openvas_scan_id, [target_aux])
1063
        # Set port range
1064
        port_range = ('port_range|||%s' % ports)
1065
        self.openvas_db.add_single_item(
1066
            'internal/%s/scanprefs' % openvas_scan_id, [port_range])
1067
1068
        # Set credentials
1069
        credentials = self.get_scan_credentials(scan_id, target)
1070
        if credentials:
1071
            cred_prefs = self.build_credentials_as_prefs(credentials)
1072
            self.openvas_db.add_single_item(
1073
                'internal/%s/scanprefs' % openvas_scan_id, cred_prefs)
1074
1075
        # Set plugins to run
1076
        nvts = self.get_scan_vts(scan_id)
1077
        if nvts != '':
1078
            nvts_list, nvts_params = self.process_vts(nvts)
1079
            # Add nvts list
1080
            separ = ';'
1081
            plugin_list = 'plugin_set|||%s' % separ.join(nvts_list)
1082
            self.openvas_db.add_single_item(
1083
                'internal/%s/scanprefs' % openvas_scan_id, [plugin_list])
1084
            # Add nvts parameters
1085
            for elem in nvts_params:
1086
                item = '%s|||%s' % (elem[0], elem[1])
1087
                self.openvas_db.add_single_item(
1088
                    'internal/%s/scanprefs' % openvas_scan_id, [item])
1089
        else:
1090
            self.openvas_db.release_db(self.main_kbindex)
1091
            self.add_scan_error(scan_id, name='', host=target,
1092
                                value='No VTS to run.')
1093
            return 2
1094
1095
        # Create a general log entry about executing OpenVAS
1096
        # It is important to send at least one result, otherwise
1097
        # the host details won't be stored.
1098
        self.add_scan_log(scan_id, host=target, name='OpenVAS summary',
1099
                          value='An OpenVAS Scanner was started for %s.'
1100
                          % target)
1101
1102
        self.add_scan_log(scan_id, host=target, name='KB location Found',
1103
                          value='KB location path was found: %s.'
1104
                          % self.openvas_db.db_address)
1105
1106
        self.add_scan_log(scan_id, host=target, name='Feed Update',
1107
                          value='Feed version: %s.'
1108
                          % self.nvti.get_feed_version())
1109
1110
        cmd = ['openvassd', '--scan-start', openvas_scan_id]
1111
        try:
1112
            result = subprocess.Popen(cmd, shell=False)
1113
        except OSError:
1114
            # the command is not available
1115
            return False
1116
1117
        ovas_pid = result.pid
1118
        logger.debug('pid = {0}'.format(ovas_pid))
1119
        self.openvas_db.add_single_item('internal/ovas_pid', [ovas_pid])
1120
1121
        # Wait until the scanner starts and loads all the preferences.
1122
        while self.openvas_db.get_single_item('internal/'+ openvas_scan_id) == 'new':
1123
            time.sleep(1)
1124
1125
        no_id_found = False
1126
        while True:
1127
            time.sleep(3)
1128
1129
            # Check if the client stopped the whole scan
1130
            if self.scan_is_stopped(openvas_scan_id):
1131
                return 1
1132
1133
            ctx = self.openvas_db.kb_connect(self.main_kbindex)
1134
            self.openvas_db.set_redisctx(ctx)
1135
            dbs = self.openvas_db.get_list_item('internal/dbindex')
1136
            for i in list(dbs):
1137
                if i == self.main_kbindex:
1138
                    continue
1139
                self.openvas_db.select_kb(ctx, str(i), set_global=True)
1140
                id_aux = self.openvas_db.get_single_item('internal/scan_id')
1141
                if not id_aux:
1142
                    continue
1143
                if id_aux == openvas_scan_id:
1144
                    no_id_found = False
1145
                    self.get_openvas_timestamp_scan_host(scan_id, target)
1146
                    self.get_openvas_result(scan_id)
1147
                    self.get_openvas_status(scan_id, target)
1148
                    if self.scan_is_finished(openvas_scan_id):
1149
                        self.openvas_db.select_kb(
1150
                            ctx, str(self.main_kbindex), set_global=False)
1151
                        self.openvas_db.remove_list_item('internal/dbindex', i)
1152
                        self.openvas_db.release_db(i)
1153
1154
            # Scan end. No kb in use for this scan id
1155
            if no_id_found:
1156
                break
1157
            no_id_found = True
1158
1159
        # Delete keys from KB related to this scan task.
1160
        self.openvas_db.release_db(self.main_kbindex)
1161
        return 1
1162
1163
1164
def main():
1165
    """ OSP openvas main function. """
1166
    daemon_main('OSPD - openvas wrapper', OSPDopenvas)
1167