JobHandler.stderr() - Code Metrics - Inspection of "ENK-101: Make getters and setters Pythonic." - SetBased/py-enarksh - Measure and Improve Code Quality continuously with Scrutinizer

Passed

Pull Request — master (#4)

by Oleg

created 2016-08-01 12:38 UTC

JobHandler.stderr() A

↳ Parent: JobHandler

Complexity

Conditions

Size

Total Lines

Duplication

Lines	0
Ratio	0 %

Code Coverage

Tests	0
CRAP Score	2

Importance

Changes

Metric	Value
cc	1
dl	0
loc	8
ccs	0
cts	2
cp	0
crap	2
rs	9.4285
c	0
b	0
f	0

"""
Enarksh

Copyright 2013-2016 Set Based IT Consultancy

Licence MIT
"""
from configparser import ConfigParser, Error
import os
import sys
import pwd
import traceback
import enarksh
from enarksh.Spawner.ChunkLogger import ChunkLogger


class JobHandler:
class SomeClass:
    def some_method(self):
        """Do x and return foo."""
    _allowed_users = []

    # ------------------------------------------------------------------------------------------------------------------
    def __init__(self, sch_id, rnd_id, user_name, args):
        """
        Creates a job handler for starting a job.

        :param int sch_id: The ID of the schedule of the job.
        :param int rnd_id: The ID of the job.
        :param str user_name: The user under which the job must run.
        :param args: The arguments for the job.
        """
        self._sch_id = sch_id
        self._rnd_id = rnd_id
        self._user_name = user_name
        self._args = args

        self.stdout_logger = ChunkLogger()
        self.stderr_logger = ChunkLogger()

        self._child_pid = -1
        self._stdout = -1
        self._stderr = -1

    # ------------------------------------------------------------------------------------------------------------------
    @property
    def pid(self):
        """
        Returns the PID of the job. If the job is finished the pid is -1.

        :rtype int: The PID of the job.
        """
        return self._child_pid

    # ------------------------------------------------------------------------------------------------------------------
    @property
    def stderr(self):
        """
        Returns the file descriptor for reading the stderr of the child process.

        :rtype int: file descriptor
        """
        return self._stderr

    # ------------------------------------------------------------------------------------------------------------------
    @property
    def stdout(self):
        """
        Returns the file descriptor for reading the stdout of the child process.

        :rtype int: file descriptor
        """
        return self._stdout

    # ------------------------------------------------------------------------------------------------------------------
    @property
    def rnd_id(self):
        """
        Returns the ID of the job.

        :rtype: int
        """
        return self._rnd_id

    # ------------------------------------------------------------------------------------------------------------------
    @property
    def sch_id(self):
        """
        Returns the ID of the schedule of the job.

        :rtype: int
        """
        return self._sch_id

    # ------------------------------------------------------------------------------------------------------------------
    def _log_job_start(self):
        """
        Logs the starting of a job.
        """
        print("Start rnd_id: %10d, %8s, %s" % (self._rnd_id, self._user_name, str(self._args)))

    # ------------------------------------------------------------------------------------------------------------------
    def _log_job_stop(self):
        """
        Logs the end of job.
        """
        print("End   rnd_id: %10d, %8s, %s" % (self._rnd_id, self._user_name, str(self._args)))

    # ------------------------------------------------------------------------------------------------------------------
    def set_job_has_finished(self):
        """
        Marks that the job has finished.
        """
        self._child_pid = -1

    # ------------------------------------------------------------------------------------------------------------------
    def get_logger_message(self, std):
        """
        Returns a message for the logger.

        :param str std: log for stdout, err for stderr

        :rtype dict[str,mixed]: The log message.
        """
        if std == 'out':
            chunk_logger = self.stdout_logger
        elif std == 'err':
            chunk_logger = self.stderr_logger
        else:
            raise Exception("Unknown output '%s'." % std)

        return {'type': 'log_file',
                'rnd_id': self._rnd_id,
                'name': std,
                'total_size': chunk_logger.get_total_log_size(),
                'filename1': chunk_logger.filename1,
                'filename2': chunk_logger.filename2}

    # ------------------------------------------------------------------------------------------------------------------
    def read(self, fd):
        """
        Reads data from the file descriptor and stores the data in a chunk logger.

        :param int fd: The file descriptor.
        """
        if fd == self._stdout:
            data = os.read(fd, 1000)
            if data == b'':
                # The pipe has been closed by the child process.
                os.close(self._stdout)
                self._stdout = -1
            else:
                self.stdout_logger.write(data)

        elif fd == self._stderr:
            data = os.read(fd, 1000)
            if data == b'':
                # The pipe has been closed by the child process.
                os.close(self._stderr)
                self._stderr = -1
            else:
                self.stdout_logger.write(data)
                self.stderr_logger.write(data)

        else:
            raise Error('Unknown file descriptor %d.' % fd)

    # ------------------------------------------------------------------------------------------------------------------
    def end_job(self):
class SomeClass:
    def some_method(self):
        """Do x and return foo."""
        self._log_job_stop()

        self.stdout_logger.close()
        self.stderr_logger.close()

    # ------------------------------------------------------------------------------------------------------------------
    @staticmethod
    def read_allowed_users():
        """
        Reads the user names under which enarksh is allowed to start processes.
        """
        config = ConfigParser()
        config.read(enarksh.HOME + '/etc/enarksh.cfg')

        JobHandler._allowed_users = config.get('spawner', 'users').split()

    # ------------------------------------------------------------------------------------------------------------------
    def start_job(self):
class SomeClass:
    def some_method(self):
        """Do x and return foo."""
        self._log_job_start()

        # Create pipes for stdout and stderr.
        pipe_stdout = os.pipe()
        pipe_stderr = os.pipe()

        self._child_pid = os.fork()
        if self._child_pid == 0:
            # Child process.
            try:
                # Close the read ends from the pipes.
                os.close(pipe_stdout[0])
                os.close(pipe_stderr[0])

                # Duplicate stdout and stderr on the pipes.
                sys.stdout.flush()
                sys.stderr.flush()
                os.dup2(pipe_stdout[1], sys.stdout.fileno())
                os.dup2(pipe_stderr[1], sys.stderr.fileno())

                # Set the effective user and group.
                if self._user_name in self._allowed_users:
                    _, _, uid, gid, _, _, _ = pwd.getpwnam(self._user_name)
                    os.setuid(0)

                    os.initgroups(self._user_name, gid)
                    os.setuid(uid)
                else:
                    raise SystemExit("Spanner is not allowed to start processes under user '%s'." % self._user_name)

                # Set variable for subprocess.
                os.putenv('ENK_RND_ID', str(self._rnd_id))
                os.putenv('ENK_SCH_ID', str(self._sch_id))

                # Replace this child process with the actual job.
                os.execv(self._args[0], self._args)

            except Exception as e:

                print('Unable to start job.', file=sys.stderr)
                print('Reason: %s' % e, file=sys.stderr)
                traceback.print_exc(file=sys.stderr)
                exit(-1)
        else:
            # Parent process.
            # Close the write ends from the pipes.
            os.close(pipe_stdout[1])
            os.close(pipe_stderr[1])

            # Remember the fds for reading the stdout and stderr from the child process.
            self._stdout = pipe_stdout[0]
            self._stderr = pipe_stderr[0]

            # Make reading from the pipes non-blocking.
            # fcntl.fcntl(self._stdout, 0)
            # fcntl.fcntl(self._stderr, 0)


# ----------------------------------------------------------------------------------------------------------------------


1			"""
2			Enarksh
3
4			Copyright 2013-2016 Set Based IT Consultancy
5
6			Licence MIT
7			"""
8			from configparser import ConfigParser, Error
9			import os
10			import sys
11			import pwd
12			import traceback
13			import enarksh
14			from enarksh.Spawner.ChunkLogger import ChunkLogger
15
16
17			class JobHandler:
			0 ignored issues – show Coding Style introduced 2016-07-27 12:34 UTC by Report Bug Copy Issue Report This class should have a docstring. The coding style of this project requires that you add a docstring to this code element. Below, you find an example for methods: class SomeClass: def some_method(self): """Do x and return foo.""" If you would like to know more about docstrings, we recommend to read PEP-257: Docstring Conventions. Loading history...
18			_allowed_users = []
19
20			# ------------------------------------------------------------------------------------------------------------------
21			def __init__(self, sch_id, rnd_id, user_name, args):
22			"""
23			Creates a job handler for starting a job.
24
25			:param int sch_id: The ID of the schedule of the job.
26			:param int rnd_id: The ID of the job.
27			:param str user_name: The user under which the job must run.
28			:param args: The arguments for the job.
29			"""
30			self._sch_id = sch_id
31			self._rnd_id = rnd_id
32			self._user_name = user_name
33			self._args = args
34
35			self.stdout_logger = ChunkLogger()
36			self.stderr_logger = ChunkLogger()
37
38			self._child_pid = -1
39			self._stdout = -1
40			self._stderr = -1
41
42			# ------------------------------------------------------------------------------------------------------------------
43			@property
44			def pid(self):
45			"""
46			Returns the PID of the job. If the job is finished the pid is -1.
47
48			:rtype int: The PID of the job.
49			"""
50			return self._child_pid
51
52			# ------------------------------------------------------------------------------------------------------------------
53			@property
54			def stderr(self):
55			"""
56			Returns the file descriptor for reading the stderr of the child process.
57
58			:rtype int: file descriptor
59			"""
60			return self._stderr
61
62			# ------------------------------------------------------------------------------------------------------------------
63			@property
64			def stdout(self):
65			"""
66			Returns the file descriptor for reading the stdout of the child process.
67
68			:rtype int: file descriptor
69			"""
70			return self._stdout
71
72			# ------------------------------------------------------------------------------------------------------------------
73			@property
74			def rnd_id(self):
75			"""
76			Returns the ID of the job.
77
78			:rtype: int
79			"""
80			return self._rnd_id
81
82			# ------------------------------------------------------------------------------------------------------------------
83			@property
84			def sch_id(self):
85			"""
86			Returns the ID of the schedule of the job.
87
88			:rtype: int
89			"""
90			return self._sch_id
91
92			# ------------------------------------------------------------------------------------------------------------------
93			def _log_job_start(self):
94			"""
95			Logs the starting of a job.
96			"""
97			print("Start rnd_id: %10d, %8s, %s" % (self._rnd_id, self._user_name, str(self._args)))
98
99			# ------------------------------------------------------------------------------------------------------------------
100			def _log_job_stop(self):
101			"""
102			Logs the end of job.
103			"""
104			print("End rnd_id: %10d, %8s, %s" % (self._rnd_id, self._user_name, str(self._args)))
105
106			# ------------------------------------------------------------------------------------------------------------------
107			def set_job_has_finished(self):
108			"""
109			Marks that the job has finished.
110			"""
111			self._child_pid = -1
112
113			# ------------------------------------------------------------------------------------------------------------------
114			def get_logger_message(self, std):
115			"""
116			Returns a message for the logger.
117
118			:param str std: log for stdout, err for stderr
119
120			:rtype dict[str,mixed]: The log message.
121			"""
122			if std == 'out':
123			chunk_logger = self.stdout_logger
124			elif std == 'err':
125			chunk_logger = self.stderr_logger
126			else:
127			raise Exception("Unknown output '%s'." % std)
128
129			return {'type': 'log_file',
130			'rnd_id': self._rnd_id,
131			'name': std,
132			'total_size': chunk_logger.get_total_log_size(),
133			'filename1': chunk_logger.filename1,
134			'filename2': chunk_logger.filename2}
135
136			# ------------------------------------------------------------------------------------------------------------------
137			def read(self, fd):
138			"""
139			Reads data from the file descriptor and stores the data in a chunk logger.
140
141			:param int fd: The file descriptor.
142			"""
143			if fd == self._stdout:
144			data = os.read(fd, 1000)
145			if data == b'':
146			# The pipe has been closed by the child process.
147			os.close(self._stdout)
148			self._stdout = -1
149			else:
150			self.stdout_logger.write(data)
151
152			elif fd == self._stderr:
153			data = os.read(fd, 1000)
154			if data == b'':
155			# The pipe has been closed by the child process.
156			os.close(self._stderr)
157			self._stderr = -1
158			else:
159			self.stdout_logger.write(data)
160			self.stderr_logger.write(data)
161
162			else:
163			raise Error('Unknown file descriptor %d.' % fd)
164
165			# ------------------------------------------------------------------------------------------------------------------
166			def end_job(self):
			0 ignored issues – show Coding Style introduced 2016-07-29 16:28 UTC by Report Bug Copy Issue Report This method should have a docstring. The coding style of this project requires that you add a docstring to this code element. Below, you find an example for methods: class SomeClass: def some_method(self): """Do x and return foo.""" If you would like to know more about docstrings, we recommend to read PEP-257: Docstring Conventions. Loading history...
167			self._log_job_stop()
168
169			self.stdout_logger.close()
170			self.stderr_logger.close()
171
172			# ------------------------------------------------------------------------------------------------------------------
173			@staticmethod
174			def read_allowed_users():
175			"""
176			Reads the user names under which enarksh is allowed to start processes.
177			"""
178			config = ConfigParser()
179			config.read(enarksh.HOME + '/etc/enarksh.cfg')
180
181			JobHandler._allowed_users = config.get('spawner', 'users').split()
182
183			# ------------------------------------------------------------------------------------------------------------------
184			def start_job(self):
			0 ignored issues – show Coding Style introduced 2016-07-27 12:34 UTC by Report Bug Copy Issue Report This method should have a docstring. The coding style of this project requires that you add a docstring to this code element. Below, you find an example for methods: class SomeClass: def some_method(self): """Do x and return foo.""" If you would like to know more about docstrings, we recommend to read PEP-257: Docstring Conventions. Loading history...
185			self._log_job_start()
186
187			# Create pipes for stdout and stderr.
188			pipe_stdout = os.pipe()
189			pipe_stderr = os.pipe()
190
191			self._child_pid = os.fork()
192			if self._child_pid == 0:
193			# Child process.
194			try:
195			# Close the read ends from the pipes.
196			os.close(pipe_stdout[0])
197			os.close(pipe_stderr[0])
198
199			# Duplicate stdout and stderr on the pipes.
200			sys.stdout.flush()
201			sys.stderr.flush()
202			os.dup2(pipe_stdout[1], sys.stdout.fileno())
203			os.dup2(pipe_stderr[1], sys.stderr.fileno())
204
205			# Set the effective user and group.
206			if self._user_name in self._allowed_users:
207			_, _, uid, gid, _, _, _ = pwd.getpwnam(self._user_name)
208			os.setuid(0)
209
210			os.initgroups(self._user_name, gid)
211			os.setuid(uid)
212			else:
213			raise SystemExit("Spanner is not allowed to start processes under user '%s'." % self._user_name)
214
215			# Set variable for subprocess.
216			os.putenv('ENK_RND_ID', str(self._rnd_id))
217			os.putenv('ENK_SCH_ID', str(self._sch_id))
218
219			# Replace this child process with the actual job.
220			os.execv(self._args[0], self._args)
221
222			except Exception as e:
			1 ignored issue – show Best Practice introduced 2016-07-27 12:34 UTC by Report Bug Copy Issue Report Catching very general exceptions such as `Exception` is usually not recommended. Generally, you would want to handle very specific errors in the exception handler. This ensure that you do not hide other types of errors which should be fixed. So, unless you specifically plan to handle any error, consider adding a more specific exception. Loading history... Coding Style Naming introduced 2016-07-27 12:34 UTC by Report Bug Copy Issue Report The name `e` does not conform to the variable naming conventions (`[a-z_][a-z0-9_]{1,60}$`). This check looks for invalid names for a range of different identifiers. You can set regular expressions to which the identifiers must conform if the defaults do not match your requirements. If your project includes a Pylint configuration file, the settings contained in that file take precedence. To find out more about Pylint, please refer to their site. Loading history...
223			print('Unable to start job.', file=sys.stderr)
224			print('Reason: %s' % e, file=sys.stderr)
225			traceback.print_exc(file=sys.stderr)
226			exit(-1)
227			else:
228			# Parent process.
229			# Close the write ends from the pipes.
230			os.close(pipe_stdout[1])
231			os.close(pipe_stderr[1])
232
233			# Remember the fds for reading the stdout and stderr from the child process.
234			self._stdout = pipe_stdout[0]
235			self._stderr = pipe_stderr[0]
236
237			# Make reading from the pipes non-blocking.
238			# fcntl.fcntl(self._stdout, 0)
239			# fcntl.fcntl(self._stderr, 0)
240
241
242			# ----------------------------------------------------------------------------------------------------------------------
243

SetBased / py-enarksh

GitHub Access Token became invalid

Pull Request — master (#4)

JobHandler.stderr() A

Complexity

Size

Duplication

Code Coverage

Importance

Duplication Side-by-Side

Filter issues like