GitHub Access Token became invalid

It seems like the GitHub access token used for retrieving details about this repository from GitHub became invalid. This might prevent certain types of inspections from being run (in particular, everything related to pull requests).
Please ask an admin of your repository to re-new the access token on this website.
Passed
Push — master ( d4663b...83e50d )
by P.R.
02:57
created

Controller.__startup()   B

Complexity

Conditions 1

Size

Total Lines 29

Duplication

Lines 0
Ratio 0 %

Code Coverage

Tests 0
CRAP Score 2

Importance

Changes 0
Metric Value
cc 1
c 0
b 0
f 0
dl 0
loc 29
rs 8.8571
ccs 0
cts 14
cp 0
crap 2
1
"""
2
Enarksh
3
4
Copyright 2013-2016 Set Based IT Consultancy
5
6
Licence MIT
7
"""
8
import gc
9
import os
10
import pwd
11
12
import zmq
13
14
import enarksh
15
from enarksh.DataLayer import DataLayer
16
from enarksh.controller import resource
17
from enarksh.controller.Schedule import Schedule
18
from enarksh.controller.event_handler.DynamicWorkerDefinitionMessageEventHandler import \
0 ignored issues
show
Bug introduced by
The name event_handler does not seem to exist in module enarksh.controller.
Loading history...
Configuration introduced by
The import enarksh.controller.event...tionMessageEventHandler could not be resolved.

This can be caused by one of the following:

1. Missing Dependencies

This error could indicate a configuration issue of Pylint. Make sure that your libraries are available by adding the necessary commands.

# .scrutinizer.yml
before_commands:
    - sudo pip install abc # Python2
    - sudo pip3 install abc # Python3
Tip: We are currently not using virtualenv to run pylint, when installing your modules make sure to use the command for the correct version.

2. Missing __init__.py files

This error could also result from missing __init__.py files in your module folders. Make sure that you place one file in each sub-folder.

Loading history...
19
    DynamicWorkerDefinitionMessageEventHandler
20
from enarksh.controller.event_handler.EventQueueEmptyEventHandler import EventQueueEmptyEventHandler
0 ignored issues
show
Bug introduced by
The name event_handler does not seem to exist in module enarksh.controller.
Loading history...
Configuration introduced by
The import enarksh.controller.event...tQueueEmptyEventHandler could not be resolved.

This can be caused by one of the following:

1. Missing Dependencies

This error could indicate a configuration issue of Pylint. Make sure that your libraries are available by adding the necessary commands.

# .scrutinizer.yml
before_commands:
    - sudo pip install abc # Python2
    - sudo pip3 install abc # Python3
Tip: We are currently not using virtualenv to run pylint, when installing your modules make sure to use the command for the correct version.

2. Missing __init__.py files

This error could also result from missing __init__.py files in your module folders. Make sure that you place one file in each sub-folder.

Loading history...
21
from enarksh.controller.event_handler.JobFinishedMessageEventHandler import JobFinishedMessageEventHandler
0 ignored issues
show
Bug introduced by
The name event_handler does not seem to exist in module enarksh.controller.
Loading history...
Configuration introduced by
The import enarksh.controller.event...shedMessageEventHandler could not be resolved.

This can be caused by one of the following:

1. Missing Dependencies

This error could indicate a configuration issue of Pylint. Make sure that your libraries are available by adding the necessary commands.

# .scrutinizer.yml
before_commands:
    - sudo pip install abc # Python2
    - sudo pip3 install abc # Python3
Tip: We are currently not using virtualenv to run pylint, when installing your modules make sure to use the command for the correct version.

2. Missing __init__.py files

This error could also result from missing __init__.py files in your module folders. Make sure that you place one file in each sub-folder.

Loading history...
22
from enarksh.controller.event_handler.RequestNodeActionMessageEventHandler import RequestNodeActionMessageEventHandler
0 ignored issues
show
Bug introduced by
The name event_handler does not seem to exist in module enarksh.controller.
Loading history...
Configuration introduced by
The import enarksh.controller.event...tionMessageEventHandler could not be resolved.

This can be caused by one of the following:

1. Missing Dependencies

This error could indicate a configuration issue of Pylint. Make sure that your libraries are available by adding the necessary commands.

# .scrutinizer.yml
before_commands:
    - sudo pip install abc # Python2
    - sudo pip3 install abc # Python3
Tip: We are currently not using virtualenv to run pylint, when installing your modules make sure to use the command for the correct version.

2. Missing __init__.py files

This error could also result from missing __init__.py files in your module folders. Make sure that you place one file in each sub-folder.

Loading history...
23
from enarksh.controller.event_handler.RequestPossibleNodeActionsMessageEventHandler import \
0 ignored issues
show
Bug introduced by
The name event_handler does not seem to exist in module enarksh.controller.
Loading history...
Configuration introduced by
The import enarksh.controller.event...ionsMessageEventHandler could not be resolved.

This can be caused by one of the following:

1. Missing Dependencies

This error could indicate a configuration issue of Pylint. Make sure that your libraries are available by adding the necessary commands.

# .scrutinizer.yml
before_commands:
    - sudo pip install abc # Python2
    - sudo pip3 install abc # Python3
Tip: We are currently not using virtualenv to run pylint, when installing your modules make sure to use the command for the correct version.

2. Missing __init__.py files

This error could also result from missing __init__.py files in your module folders. Make sure that you place one file in each sub-folder.

Loading history...
24
    RequestPossibleNodeActionsMessageEventHandler
25
from enarksh.controller.event_handler.ScheduleDefinitionMessageEventHandler import ScheduleDefinitionMessageEventHandler
0 ignored issues
show
Bug introduced by
The name event_handler does not seem to exist in module enarksh.controller.
Loading history...
Configuration introduced by
The import enarksh.controller.event...tionMessageEventHandler could not be resolved.

This can be caused by one of the following:

1. Missing Dependencies

This error could indicate a configuration issue of Pylint. Make sure that your libraries are available by adding the necessary commands.

# .scrutinizer.yml
before_commands:
    - sudo pip install abc # Python2
    - sudo pip3 install abc # Python3
Tip: We are currently not using virtualenv to run pylint, when installing your modules make sure to use the command for the correct version.

2. Missing __init__.py files

This error could also result from missing __init__.py files in your module folders. Make sure that you place one file in each sub-folder.

Loading history...
26
from enarksh.controller.message.DynamicWorkerDefinitionMessage import DynamicWorkerDefinitionMessage
27
from enarksh.controller.message.JobFinishedMessage import JobFinishedMessage
28
from enarksh.controller.message.RequestNodeActionMessage import RequestNodeActionMessage
29
from enarksh.controller.message.RequestPossibleNodeActionsMessage import RequestPossibleNodeActionsMessage
30
from enarksh.controller.message.ScheduleDefinitionMessage import ScheduleDefinitionMessage
31
from enarksh.event.EventActor import EventActor
32
from enarksh.event.EventController import EventController
33
from enarksh.message.MessageController import MessageController
0 ignored issues
show
Bug introduced by
The name message does not seem to exist in module enarksh.
Loading history...
Configuration introduced by
The import enarksh.message.MessageController could not be resolved.

This can be caused by one of the following:

1. Missing Dependencies

This error could indicate a configuration issue of Pylint. Make sure that your libraries are available by adding the necessary commands.

# .scrutinizer.yml
before_commands:
    - sudo pip install abc # Python2
    - sudo pip3 install abc # Python3
Tip: We are currently not using virtualenv to run pylint, when installing your modules make sure to use the command for the correct version.

2. Missing __init__.py files

This error could also result from missing __init__.py files in your module folders. Make sure that you place one file in each sub-folder.

Loading history...
34
35
36
class Controller(EventActor):
0 ignored issues
show
Coding Style introduced by
This class should have a docstring.

The coding style of this project requires that you add a docstring to this code element. Below, you find an example for methods:

class SomeClass:
    def some_method(self):
        """Do x and return foo."""

If you would like to know more about docstrings, we recommend to read PEP-257: Docstring Conventions.

Loading history...
37
    # ------------------------------------------------------------------------------------------------------------------
38
    def __init__(self):
39
        """
40
        Object constructor.
41
        """
42
        self.event_controller = EventController()
43
        """
44
        The event controller.
45
46
        :type: enarksh.event.EventController.EventController
47
        """
48
49
        EventActor.__init__(self)
50
51
        self.message_controller = MessageController()
52
        """
53
        The message controller.
54
55
        :type: enarksh.message.MessageController.MessageController
56
        """
57
58
        self.host_resources = {}
59
        """
60
        All resources defined at host level.
61
62
        :type: dict
63
        """
64
65
        self.schedules = {}
66
        """
67
        All the current schedules.
68
69
        :type: dict[int,enarksh.controller.Schedule.Schedule]
70
        """
71
72
    # ------------------------------------------------------------------------------------------------------------------
73
    @staticmethod
74
    def daemonize():
0 ignored issues
show
Coding Style introduced by
This method should have a docstring.

The coding style of this project requires that you add a docstring to this code element. Below, you find an example for methods:

class SomeClass:
    def some_method(self):
        """Do x and return foo."""

If you would like to know more about docstrings, we recommend to read PEP-257: Docstring Conventions.

Loading history...
75
        enarksh.daemonize(os.path.join(enarksh.HOME, 'var/lock/controllerd.pid'),
76
                          '/dev/null',
77
                          os.path.join(enarksh.HOME, 'var/log/controllerd.log'),
78
                          os.path.join(enarksh.HOME, 'var/log/controllerd.log'))
79
80
    # ------------------------------------------------------------------------------------------------------------------
81
    @staticmethod
82
    def __set_unprivileged_user():
83
        """
84
        Set the real and effective user and group to an unprivileged user.
85
        """
86
        _, _, uid, gid, _, _, _ = pwd.getpwnam('enarksh')
87
88
        os.setresgid(gid, gid, 0)
89
        os.setresuid(uid, uid, 0)
90
91
    # ------------------------------------------------------------------------------------------------------------------
92
    def __create_host_resources(self):
93
        """
94
        Creates resources defined at host level.
95
        """
96
        resources_data = DataLayer.enk_back_get_host_resources()
97
        for resource_data in resources_data:
98
            self.host_resources[resource_data['rsc_id']] = resource.create_resource(resource_data)
99
100
    # ------------------------------------------------------------------------------------------------------------------
101
    def __startup(self):
102
        """
103
        Performs the necessary actions for starting the controller daemon.
104
        """
105
        print('Start controller.')
106
107
        # Set database configuration options.
108
        DataLayer.config['host'] = enarksh.MYSQL_HOSTNAME
109
        DataLayer.config['user'] = enarksh.MYSQL_USERNAME
110
        DataLayer.config['password'] = enarksh.MYSQL_PASSWORD
111
        DataLayer.config['database'] = enarksh.MYSQL_SCHEMA
112
        DataLayer.config['port'] = enarksh.MYSQL_PORT
113
        DataLayer.config['autocommit'] = False
114
115
        # Connect to the MySQL.
116
        DataLayer.connect()
117
        DataLayer.start_transaction()
118
119
        # Sanitise the data in the database.
120
        DataLayer.enk_back_controller_init()
121
122
        # Create resources defined at host level.
123
        self.__create_host_resources()
124
125
        # Set the effective user and group to an unprivileged user and group.
126
        self.__set_unprivileged_user()
127
128
        # Commit transaction and close connection to MySQL.
129
        DataLayer.commit()
130
131
    # ------------------------------------------------------------------------------------------------------------------
132
    @staticmethod
133
    def __shutdown():
134
        """
135
        Performs the necessary actions for stopping the controller.
136
        """
137
        # Log stop of the controller.
138
        print('Stop controller')
139
140
    # ------------------------------------------------------------------------------------------------------------------
141
    def slot_schedule_termination(self, event, rst_id, _listener_data):
142
        """
143
144
        :param enarksh.event.Event.Event event: The event.
145
        :param int rst_id: The run status of the schedule.
146
        :param * _listener_data: Not used
147
        """
148
        del _listener_data
149
150
        schedule = event.source
151
152
        print("Schedule %s has terminated with status %s" % (schedule.sch_id, rst_id))
153
154
        self.unload_schedule(schedule.sch_id)
155
156
    # ------------------------------------------------------------------------------------------------------------------
157
    def load_schedule(self, sch_id):
158
        """
159
        :param int sch_id:
160
161
        :rtype: enarksh.controller.Schedule.Schedule
162
        """
163
        print("Loading schedule '%s'." % sch_id)
164
165
        # Load the schedule.
166
        schedule = Schedule(sch_id, self.host_resources)
167
        schedule.event_schedule_termination.register_listener(self.slot_schedule_termination)
168
169
        # Register the schedule.
170
        self.schedules[sch_id] = schedule
171
172
        return schedule
173
174
    # ------------------------------------------------------------------------------------------------------------------
175
    def unload_schedule(self, sch_id):
176
        """
177
        :param int sch_id:
178
        """
179
        print("Unloading schedule '%s'." % sch_id)
180
181
        if sch_id in self.schedules:
182
            # Remove the schedule.
183
            del self.schedules[sch_id]
184
185
            gc.collect()
186
187
    # ------------------------------------------------------------------------------------------------------------------
188
    def reload_schedule(self, sch_id):
189
        """
190
        :param int sch_id:
191
192
        :rtype: enarksh.controller.Schedule.Schedule
193
        """
194
        self.unload_schedule(sch_id)
195
196
        return self.load_schedule(sch_id)
197
198
    # ------------------------------------------------------------------------------------------------------------------
199
    def get_schedule_by_sch_id(self, sch_id):
200
        """
201
        Returns a schedule.
202
203
        :param int sch_id: The ID of the schedule.
204
205
        :rtype: enarksh.controller.Schedule.Schedule
206
        """
207
        schedule = self.schedules.get(int(sch_id), None)
208
        if not schedule:
209
            # Load the schedule if the schedule is not currently loaded.
210
            schedule = self.load_schedule(sch_id)
211
212
        return schedule
213
214
    # ------------------------------------------------------------------------------------------------------------------
215
    def _register_sockets(self):
216
        """
217
        Registers ZMQ sockets for communication with other processes in Enarksh.
218
        """
219
        # Register socket for receiving asynchronous incoming messages.
220
        self.message_controller.register_end_point('pull', zmq.PULL, enarksh.CONTROLLER_PULL_END_POINT)
1 ignored issue
show
Bug introduced by
The Module zmq does not seem to have a member named PULL.

This check looks for calls to members that are non-existent. These calls will fail.

The member could have been renamed or removed.

Loading history...
221
222
        # Create socket for lockstep incoming messages.       .
223
        self.message_controller.register_end_point('lockstep', zmq.REP, enarksh.CONTROLLER_LOCKSTEP_END_POINT)
1 ignored issue
show
Bug introduced by
The Module zmq does not seem to have a member named REP.

This check looks for calls to members that are non-existent. These calls will fail.

The member could have been renamed or removed.

Loading history...
224
225
        # Create socket for sending asynchronous messages to the spanner.
226
        self.message_controller.register_end_point('spawner', zmq.PUSH, enarksh.SPAWNER_PULL_END_POINT)
227
228
        # Create socket for sending asynchronous messages to the logger.
229
        self.message_controller.register_end_point('logger', zmq.PUSH, enarksh.LOGGER_PULL_END_POINT)
230
231
    # ------------------------------------------------------------------------------------------------------------------
232
    def _register_message_types(self):
233
        """
234
        Registers all message type that the controller handles at the message controller.
235
        """
236
        self.message_controller.register_message_type(DynamicWorkerDefinitionMessage.MESSAGE_TYPE)
237
        self.message_controller.register_message_type(JobFinishedMessage.MESSAGE_TYPE)
238
        self.message_controller.register_message_type(RequestNodeActionMessage.MESSAGE_TYPE)
239
        self.message_controller.register_message_type(RequestPossibleNodeActionsMessage.MESSAGE_TYPE)
240
        self.message_controller.register_message_type(ScheduleDefinitionMessage.MESSAGE_TYPE)
241
242
    # ------------------------------------------------------------------------------------------------------------------
243
    def _register_events_handlers(self):
244
        """
245
        Registers all event handlers at the event controller.
246
        """
247
        # Register message received event handlers.
248
        self.message_controller.register_listener(DynamicWorkerDefinitionMessage.MESSAGE_TYPE,
249
                                                  DynamicWorkerDefinitionMessageEventHandler.handle,
250
                                                  self)
251
        self.message_controller.register_listener(JobFinishedMessage.MESSAGE_TYPE,
252
                                                  JobFinishedMessageEventHandler.handle,
253
                                                  self)
254
        self.message_controller.register_listener(RequestNodeActionMessage.MESSAGE_TYPE,
255
                                                  RequestNodeActionMessageEventHandler.handle,
256
                                                  self)
257
        self.message_controller.register_listener(RequestPossibleNodeActionsMessage.MESSAGE_TYPE,
258
                                                  RequestPossibleNodeActionsMessageEventHandler.handle,
259
                                                  self)
260
        self.message_controller.register_listener(ScheduleDefinitionMessage.MESSAGE_TYPE,
261
                                                  ScheduleDefinitionMessageEventHandler.handle,
262
                                                  self)
263
264
        # Register other event handlers.
265
        self.event_controller.event_queue_empty.register_listener(EventQueueEmptyEventHandler.handle, self)
266
267
    # ------------------------------------------------------------------------------------------------------------------
268
    def main(self):
269
        """
270
        The main function of the job spawner.
271
        """
272
        self.__startup()
273
274
        self._register_sockets()
275
276
        self._register_message_types()
277
278
        self._register_events_handlers()
279
280
        self.message_controller.no_barking(5)
281
282
        self.event_controller.loop()
283
284
        self.__shutdown()
285
286
# ----------------------------------------------------------------------------------------------------------------------
287