run_local_bears_on_file() - Code Metrics - Inspection of "WIP: Udayan/newbears" - coala-analyzer/coala - Measure and Improve Code Quality continuously with Scrutinizer

Completed

Pull Request — master (#2609)

by Udayan

created 2016-08-21 18:05 UTC

run_local_bears_on_file() B

↳ Parent: Project

Complexity

Conditions

Size

Total Lines

Duplication

Lines	0
Ratio	0 %

Importance

Changes	1
Bugs	0	Features	0

Metric	Value
cc	3
c	1
b	0
f	0
dl	0
loc	43
rs	8.8571

import queue
import traceback

from coalib.bears.BEAR_KIND import BEAR_KIND
from coalib.bears.GlobalBear import GlobalBear
from coalib.bears.LocalBear import LocalBear
from coalib.misc import Constants
from coalib.processes.communication.LogMessage import LOG_LEVEL, LogMessage
from coalib.processes.CONTROL_ELEMENT import CONTROL_ELEMENT
from coalib.results.Result import Result


def send_msg(message_queue, timeout, log_level, *args, delimiter=' ', end=''):
    """
    Puts message into message queue for a LogPrinter to present to the user.

    :param message_queue: The queue to put the message into and which the
                          LogPrinter reads.
    :param timeout:       The queue blocks at most timeout seconds for a free
                          slot to execute the put operation on. After the
                          timeout it returns queue Full exception.
    :param log_level:     The log_level i.e Error,Debug or Warning.It is sent
                          to the LogPrinter depending on the message.
    :param args:          This includes the elements of the message.
    :param delimiter:     It is the value placed between each arg. By default
                          it is a ' '.
    :param end:           It is the value placed at the end of the message.
    """
    output = str(delimiter).join(str(arg) for arg in args) + str(end)
    message_queue.put(LogMessage(log_level, output),
                      timeout=timeout)


def validate_results(message_queue, timeout, result_list, name, args, kwargs):
    """
    Validates if the result_list passed to it contains valid set of results.
    That is the result_list must itself be a list and contain objects of the
    instance of Result object. If any irregularity is found a message is put in
    the message_queue to present the irregularity to the user. Each result_list
    belongs to an execution of a bear.

    :param message_queue: A queue that contains messages of type
                          errors/warnings/debug statements to be printed in the
                          Log.
    :param timeout:       The queue blocks at most timeout seconds for a free
                          slot to execute the put operation on. After the
                          timeout it returns queue Full exception.
    :param result_list:   The list of results to validate.
    :param name:          The name of the bear executed.
    :param args:          The args with which the bear was executed.
    :param kwargs:        The kwargs with which the bear was executed.
    :return:              Returns None if the result_list is invalid. Else it
                          returns the result_list itself.
    """
    if result_list is None:
        return None

    for result in result_list:
        if not isinstance(result, Result):
            send_msg(message_queue,
                     timeout,
                     LOG_LEVEL.ERROR,
                     "The results from the bear {bear} could only be "
                     "partially processed with arguments {arglist}, "
                     "{kwarglist}"
                     .format(bear=name, arglist=args, kwarglist=kwargs))
            send_msg(message_queue,
                     timeout,
                     LOG_LEVEL.DEBUG,
                     "One of the results in the list for the bear {bear} is "
                     "an instance of {ret} but it should be an instance of "
                     "Result"
                     .format(bear=name, ret=result.__class__))
            result_list.remove(result)

    return result_list


def run_bear(message_queue, timeout, bear_instance, *args, **kwargs):
    """
    This method is responsible for executing the instance of a bear. It also
    reports or logs errors if any occur during the execution of that bear
    instance.

    :param message_queue: A queue that contains messages of type
                          errors/warnings/debug statements to be printed in the
                          Log.
    :param timeout:       The queue blocks at most timeout seconds for a free
                          slot to execute the put operation on. After the
                          timeout it returns queue Full exception.
    :param bear_instance: The instance of the bear to be executed.
    :param args:          The arguments that are to be passed to the bear.
    :param kwargs:        The keyword arguments that are to be passed to the
                          bear.
    :return:              Returns a valid list of objects of the type Result
                          if the bear executed successfully. None otherwise.
    """
    if kwargs.get("dependency_results", True) is None:
        del kwargs["dependency_results"]

    name = bear_instance.name

    try:
        result_list = bear_instance.execute(*args, **kwargs)
    except:
        send_msg(message_queue,
                 timeout,
                 LOG_LEVEL.ERROR,
                 "The bear {bear} failed to run with the arguments "
                 "{arglist}, {kwarglist}. Skipping bear..."
                 .format(bear=name, arglist=args, kwarglist=kwargs))
        send_msg(message_queue,
                 timeout,
                 LOG_LEVEL.DEBUG,
                 "Traceback for error in bear {}:".format(name),
                 traceback.format_exc(),
                 delimiter="\n")

        return None

    return validate_results(message_queue,
                            timeout,
                            result_list,
                            name,
                            args,
                            kwargs)


def get_local_dependency_results(local_result_list, bear_instance):
    """
    This method gets all the results originating from the dependencies of a
    bear_instance. Each bear_instance may or may not have dependencies.

    :param local_result_list: The list of results out of which the dependency
                              results are picked.
    :param bear_instance:     The instance of a local bear to get the
                              dependencies from.
    :return:                  Return none if there are no dependencies for the
                              bear. Else return a dictionary containing
                              dependency results.
    """
    deps = bear_instance.BEAR_DEPS
    if not deps:
        return None

    dependency_results = {}
    dep_strings = []
    for dep in deps:
        dep_strings.append(dep.__name__)

    for result in local_result_list:
        if result.origin in dep_strings:
            results = dependency_results.get(result.origin, [])
            results.append(result)
            dependency_results[result.origin] = results

    return dependency_results


def run_local_bear(message_queue,
                   timeout,
                   local_result_list,
                   bear_instance,
                   filename,
                   file_contents):
    """
    Runs an instance of a local bear. Checks if bear_instance is of type
    LocalBear and then passes it to the run_bear to execute.

    :param message_queue:     A queue that contains messages of type
                              errors/warnings/debug statements to be printed in
                              the Log.
    :param timeout:           The queue blocks at most timeout seconds for a
                              free slot to execute the put operation on. After
                              the timeout it returns queue Full exception.
    :param local_result_list: Its a list that stores the results of all local
                              bears.
    :param bear_instance:     Instance of LocalBear the run.
    :param filename:          Name of the file to run it on.
    :param file_contents:     List that contains contents of the file line by
                              line.
    :return:                  Returns a list of results generated by the passed
                              bear_instance.
    """
    if (not isinstance(bear_instance, LocalBear) or
            bear_instance.kind() != BEAR_KIND.LOCAL):
        send_msg(message_queue,
                 timeout,
                 LOG_LEVEL.WARNING,
                 "A given local bear ({}) is not valid. Leaving "
                 "it out...".format(bear_instance.__class__.__name__),
                 Constants.THIS_IS_A_BUG)

        return None

    kwargs = {"dependency_results":
              get_local_dependency_results(local_result_list,
                                           bear_instance)}
    return run_bear(message_queue,
                    timeout,
                    bear_instance,
                    filename,
                    file_contents,
                    **kwargs)


def run_global_bear(message_queue,
                    timeout,
                    global_bear_instance,
                    dependency_results):
    """
    Runs an instance of a global bear. Checks if bear_instance is of type
    GlobalBear and then passes it to the run_bear to execute.

    :param message_queue:        A queue that contains messages of type
                                 errors/warnings/debug statements to be printed
                                 in the Log.
    :param timeout:              The queue blocks at most timeout seconds for a
                                 free slot to execute the put operation on.
                                 After the timeout it returns queue Full
                                 exception.
    :param global_bear_instance: Instance of GlobalBear to run.
    :param dependency_results:   The results of all the bears on which the
                                 instance of the passed bear to be run depends
                                 on.
    :return:                     Returns a list of results generated by the
                                 passed bear_instance.
    """
    if (not isinstance(global_bear_instance, GlobalBear)
            or global_bear_instance.kind() != BEAR_KIND.GLOBAL):
        send_msg(message_queue,
                 timeout,
                 LOG_LEVEL.WARNING,
                 "A given global bear ({}) is not valid. Leaving it "
                 "out..."
                 .format(global_bear_instance.__class__.__name__),
                 Constants.THIS_IS_A_BUG)

        return None

    kwargs = {"dependency_results": dependency_results}
    return run_bear(message_queue,
                    timeout,
                    global_bear_instance,
                    **kwargs)


def run_local_bears_on_file(message_queue,
                            timeout,
                            local_bear_list,
                            local_result_dict,
                            control_queue,
                            filename,
                            file_contents):
    """
    This method runs a list of local bears on one file.

    :param message_queue:     A queue that contains messages of type
                              errors/warnings/debug statements to be printed
                              in the Log.
    :param timeout:           The queue blocks at most timeout seconds for a
                              free slot to execute the put operation on. After
                              the timeout it returns queue Full exception.
    :param local_bear_list:   List of local bears to run on file.
    :param local_result_dict: A Manager.dict that will be used to store local
                              bear results. A list of all local bear results
                              will be stored with the filename as key.
    :param control_queue:     If any result gets written to the result_dict a
                              tuple containing a CONTROL_ELEMENT (to indicate
                              what kind of event happened) and either a bear
                              name(for global results) or a file name to
                              indicate the result will be put to the queue.
    :param filename:          The name of file on which to run the bears.
    :param file_contents      List that contains contents of the file line by
                              line.
    """

    local_result_list = []
    for bear_instance in local_bear_list:
        result = run_local_bear(message_queue,
                                timeout,
                                local_result_list,
                                bear_instance,
                                filename,
                                file_contents)
        if result is not None:
            local_result_list.extend(result)

    local_result_dict[filename] = local_result_list
    control_queue.put((CONTROL_ELEMENT.LOCAL, filename))


def get_global_dependency_results(global_result_dict, bear_instance):
    """
    This method gets all the results originating from the dependencies of a
    bear_instance. Each bear_instance may or may not have dependencies.

    :param global_result_dict: The list of results out of which the dependency
                               results are picked.
    :return:                   None if bear has no dependencies, False if
                               dependencies are not met, the dependency dict
                               otherwise.
    """
    try:
        deps = bear_instance.BEAR_DEPS
        if not deps:
            return None
    except AttributeError:
        # When this occurs we have an invalid bear and a warning will be
        # emitted later.
        return None

    dependency_results = {}
    for dep in deps:
        depname = dep.__name__
        if depname not in global_result_dict:
            return False

        dependency_results[depname] = global_result_dict[depname]

    return dependency_results


def get_next_global_bear(timeout,
                         global_bear_queue,
                         global_bear_list,
                         global_result_dict):
    """
    Retrieves the next global bear.

    :param timeout:            The queue blocks at most timeout seconds for a
                               free slot to execute the put operation on. After
                               the timeout it returns queue Full exception.
    :param global_bear_queue:  queue (read, write) of indexes of global bear
                               instances in the global_bear_list.
    :param global_bear_list:   A list containing all global bears to be
                               executed.
    :param global_result_dict: A Manager.dict that will be used to store global
                               results. The list of results of one global bear
                               will be stored with the bear name as key.
    :return:                   (bear, bearname, dependency_results)
    """
    dependency_results = False

    while dependency_results is False:
        bear_id = global_bear_queue.get(timeout=timeout)
        bear = global_bear_list[bear_id]

        dependency_results = (
            get_global_dependency_results(global_result_dict, bear))
        if dependency_results is False:
            global_bear_queue.put(bear_id)

    return bear, dependency_results


def task_done(obj):
    """
    Invokes task_done if the given queue provides this operation. Otherwise
    passes silently.

    :param obj: Any object.
    """
    if hasattr(obj, "task_done"):
        obj.task_done()


def run_local_bears(filename_queue,
                    message_queue,
                    timeout,
                    file_dict,
                    local_bear_list,
                    local_result_dict,
                    control_queue):
    """
    Run local bears on all the files given.

    :param filename_queue:    queue (read) of file names to check with
                              local bears.
    :param message_queue:     A queue that contains messages of type
                              errors/warnings/debug statements to be printed
                              in the Log.
    :param timeout:           The queue blocks at most timeout seconds for a
                              free slot to execute the put operation on. After
                              the timeout it returns queue Full exception.
    :param file_dict:         Dictionary that contains fileproxy objects.
    :param local_bear_list:   List of local bears to run.
    :param local_result_dict: A Manager.dict that will be used to store local
                              bear results. A list of all local bear results
                              will be stored with the filename as key.
    :param control_queue:     If any result gets written to the result_dict a
                              tuple containing a CONTROL_ELEMENT (to indicate
                              what kind of event happened) and either a bear
                              name(for global results) or a file name to
                              indicate the result will be put to the queue.
    """
    try:
        while True:
            filename = filename_queue.get(timeout=timeout)

            run_local_bears_on_file(message_queue,
                                    timeout,
                                    local_bear_list,
                                    local_result_dict,
                                    control_queue,
                                    filename,
                                    file_dict[filename].lines)
            task_done(filename_queue)
    except queue.Empty:
        return


def run_global_bears(message_queue,
                     timeout,
                     global_bear_queue,
                     global_bear_list,
                     global_result_dict,
                     control_queue):
    """
    Run all global bears.

    :param message_queue:      A queue that contains messages of type
                               errors/warnings/debug statements to be printed
                               in the Log.
    :param timeout:            The queue blocks at most timeout seconds for a
                               free slot to execute the put operation on. After
                               the timeout it returns queue Full exception.
    :param global_bear_queue:  queue (read, write) of indexes of global bear
                               instances in the global_bear_list.
    :param global_bear_list:   list of global bear instances
    :param global_result_dict: A Manager.dict that will be used to store global
                               results. The list of results of one global bear
                               will be stored with the bear name as key.
    :param control_queue:      If any result gets written to the result_dict a
                               tuple containing a CONTROL_ELEMENT (to indicate
                               what kind of event happened) and either a bear
                               name(for global results) or a file name to
                               indicate the result will be put to the queue.
    """
    try:
        while True:
            bear, dep_results = (
                get_next_global_bear(timeout,
                                     global_bear_queue,
                                     global_bear_list,
                                     global_result_dict))
            bearname = bear.__class__.__name__
            result = run_global_bear(message_queue, timeout, bear, dep_results)
            if result:
                global_result_dict[bearname] = result
                control_queue.put((CONTROL_ELEMENT.GLOBAL, bearname))
            else:
                global_result_dict[bearname] = None
            task_done(global_bear_queue)
    except queue.Empty:
        return


def run(file_name_queue,
        local_bear_list,
        global_bear_list,
        global_bear_queue,
        file_dict,
        local_result_dict,
        global_result_dict,
        message_queue,
        control_queue,
        timeout=0):
    """
    This is the method that is actually runs by processes.

    If parameters type is 'queue (read)' this means it has to implement the
    get(timeout=TIMEOUT) method and it shall raise queue.Empty if the queue
    is empty up until the end of the timeout. If the queue has the
    (optional!) task_done() attribute, the run method will call it after
    processing each item.

    If parameters type is 'queue (write)' it shall implement the
    put(object, timeout=TIMEOUT) method.

    If the queues raise any exception not specified here the user will get
    an 'unknown error' message. So beware of that.

    :param file_name_queue:    queue (read) of file names to check with local
                               bears. Each invocation of the run method needs
                               one such queue which it checks with all the
                               local bears. The queue could be empty.
                               (Repeat until queue empty.)
    :param local_bear_list:    List of local bear instances.
    :param global_bear_list:   List of global bear instances.
    :param global_bear_queue:  queue (read, write) of indexes of global bear
                               instances in the global_bear_list.
    :param file_dict:          dict of all files as {filename:file}, file as in
                               the fileproxy object.
    :param local_result_dict:  A Manager.dict that will be used to store local
                               results. A list of all local results.
                               will be stored with the filename as key.
    :param global_result_dict: A Manager.dict that will be used to store global
                               results. The list of results of one global bear
                               will be stored with the bear name as key.
    :param message_queue:      queue (write) for debug/warning/error
                               messages (type LogMessage)
    :param control_queue:      queue (write). If any result gets written to the
                               result_dict a tuple containing a CONTROL_ELEMENT
                               (to indicate what kind of event happened) and
                               either a bear name (for global results) or a
                               file name to indicate the result will be put to
                               the queue. If the run method finished all its
                               local bears it will put
                               (CONTROL_ELEMENT.LOCAL_FINISHED, None) to the
                               queue, if it finished all global ones,
                               (CONTROL_ELEMENT.GLOBAL_FINISHED, None) will
                               be put there.
    :param timeout:            The queue blocks at most timeout seconds for a
                               free slot to execute the put operation on. After
                               the timeout it returns queue Full exception.
    """
    try:
        run_local_bears(file_name_queue,
                        message_queue,
                        timeout,
                        file_dict,
                        local_bear_list,
                        local_result_dict,
                        control_queue)
        control_queue.put((CONTROL_ELEMENT.LOCAL_FINISHED, None))

        run_global_bears(message_queue,
                         timeout,
                         global_bear_queue,
                         global_bear_list,
                         global_result_dict,
                         control_queue)
        control_queue.put((CONTROL_ELEMENT.GLOBAL_FINISHED, None))
    except (OSError, KeyboardInterrupt):  # pragma: no cover
        pass


1			import queue
2			import traceback
3
4			from coalib.bears.BEAR_KIND import BEAR_KIND
5			from coalib.bears.GlobalBear import GlobalBear
6			from coalib.bears.LocalBear import LocalBear
7			from coalib.misc import Constants
8			from coalib.processes.communication.LogMessage import LOG_LEVEL, LogMessage
9			from coalib.processes.CONTROL_ELEMENT import CONTROL_ELEMENT
10			from coalib.results.Result import Result
11
12
13			def send_msg(message_queue, timeout, log_level, *args, delimiter=' ', end=''):
14			"""
15			Puts message into message queue for a LogPrinter to present to the user.
16
17			:param message_queue: The queue to put the message into and which the
18			LogPrinter reads.
19			:param timeout: The queue blocks at most timeout seconds for a free
20			slot to execute the put operation on. After the
21			timeout it returns queue Full exception.
22			:param log_level: The log_level i.e Error,Debug or Warning.It is sent
23			to the LogPrinter depending on the message.
24			:param args: This includes the elements of the message.
25			:param delimiter: It is the value placed between each arg. By default
26			it is a ' '.
27			:param end: It is the value placed at the end of the message.
28			"""
29			output = str(delimiter).join(str(arg) for arg in args) + str(end)
30			message_queue.put(LogMessage(log_level, output),
31			timeout=timeout)
32
33
34			def validate_results(message_queue, timeout, result_list, name, args, kwargs):
35			"""
36			Validates if the result_list passed to it contains valid set of results.
37			That is the result_list must itself be a list and contain objects of the
38			instance of Result object. If any irregularity is found a message is put in
39			the message_queue to present the irregularity to the user. Each result_list
40			belongs to an execution of a bear.
41
42			:param message_queue: A queue that contains messages of type
43			errors/warnings/debug statements to be printed in the
44			Log.
45			:param timeout: The queue blocks at most timeout seconds for a free
46			slot to execute the put operation on. After the
47			timeout it returns queue Full exception.
48			:param result_list: The list of results to validate.
49			:param name: The name of the bear executed.
50			:param args: The args with which the bear was executed.
51			:param kwargs: The kwargs with which the bear was executed.
52			:return: Returns None if the result_list is invalid. Else it
53			returns the result_list itself.
54			"""
55			if result_list is None:
56			return None
57
58			for result in result_list:
59			if not isinstance(result, Result):
60			send_msg(message_queue,
61			timeout,
62			LOG_LEVEL.ERROR,
63			"The results from the bear {bear} could only be "
64			"partially processed with arguments {arglist}, "
65			"{kwarglist}"
66			.format(bear=name, arglist=args, kwarglist=kwargs))
67			send_msg(message_queue,
68			timeout,
69			LOG_LEVEL.DEBUG,
70			"One of the results in the list for the bear {bear} is "
71			"an instance of {ret} but it should be an instance of "
72			"Result"
73			.format(bear=name, ret=result.__class__))
74			result_list.remove(result)
75
76			return result_list
77
78
79			def run_bear(message_queue, timeout, bear_instance, args, *kwargs):
80			"""
81			This method is responsible for executing the instance of a bear. It also
82			reports or logs errors if any occur during the execution of that bear
83			instance.
84
85			:param message_queue: A queue that contains messages of type
86			errors/warnings/debug statements to be printed in the
87			Log.
88			:param timeout: The queue blocks at most timeout seconds for a free
89			slot to execute the put operation on. After the
90			timeout it returns queue Full exception.
91			:param bear_instance: The instance of the bear to be executed.
92			:param args: The arguments that are to be passed to the bear.
93			:param kwargs: The keyword arguments that are to be passed to the
94			bear.
95			:return: Returns a valid list of objects of the type Result
96			if the bear executed successfully. None otherwise.
97			"""
98			if kwargs.get("dependency_results", True) is None:
99			del kwargs["dependency_results"]
100
101			name = bear_instance.name
102
103			try:
104			result_list = bear_instance.execute(args, *kwargs)
105			except:
106			send_msg(message_queue,
107			timeout,
108			LOG_LEVEL.ERROR,
109			"The bear {bear} failed to run with the arguments "
110			"{arglist}, {kwarglist}. Skipping bear..."
111			.format(bear=name, arglist=args, kwarglist=kwargs))
112			send_msg(message_queue,
113			timeout,
114			LOG_LEVEL.DEBUG,
115			"Traceback for error in bear {}:".format(name),
116			traceback.format_exc(),
117			delimiter="\n")
118
119			return None
120
121			return validate_results(message_queue,
122			timeout,
123			result_list,
124			name,
125			args,
126			kwargs)
127
128
129			def get_local_dependency_results(local_result_list, bear_instance):
130			"""
131			This method gets all the results originating from the dependencies of a
132			bear_instance. Each bear_instance may or may not have dependencies.
133
134			:param local_result_list: The list of results out of which the dependency
135			results are picked.
136			:param bear_instance: The instance of a local bear to get the
137			dependencies from.
138			:return: Return none if there are no dependencies for the
139			bear. Else return a dictionary containing
140			dependency results.
141			"""
142			deps = bear_instance.BEAR_DEPS
143			if not deps:
144			return None
145
146			dependency_results = {}
147			dep_strings = []
148			for dep in deps:
149			dep_strings.append(dep.__name__)
150
151			for result in local_result_list:
152			if result.origin in dep_strings:
153			results = dependency_results.get(result.origin, [])
154			results.append(result)
155			dependency_results[result.origin] = results
156
157			return dependency_results
158
159
160			def run_local_bear(message_queue,
161			timeout,
162			local_result_list,
163			bear_instance,
164			filename,
165			file_contents):
166			"""
167			Runs an instance of a local bear. Checks if bear_instance is of type
168			LocalBear and then passes it to the run_bear to execute.
169
170			:param message_queue: A queue that contains messages of type
171			errors/warnings/debug statements to be printed in
172			the Log.
173			:param timeout: The queue blocks at most timeout seconds for a
174			free slot to execute the put operation on. After
175			the timeout it returns queue Full exception.
176			:param local_result_list: Its a list that stores the results of all local
177			bears.
178			:param bear_instance: Instance of LocalBear the run.
179			:param filename: Name of the file to run it on.
180			:param file_contents: List that contains contents of the file line by
181			line.
182			:return: Returns a list of results generated by the passed
183			bear_instance.
184			"""
185			if (not isinstance(bear_instance, LocalBear) or
186			bear_instance.kind() != BEAR_KIND.LOCAL):
187			send_msg(message_queue,
188			timeout,
189			LOG_LEVEL.WARNING,
190			"A given local bear ({}) is not valid. Leaving "
191			"it out...".format(bear_instance.__class__.__name__),
192			Constants.THIS_IS_A_BUG)
193
194			return None
195
196			kwargs = {"dependency_results":
197			get_local_dependency_results(local_result_list,
198			bear_instance)}
199			return run_bear(message_queue,
200			timeout,
201			bear_instance,
202			filename,
203			file_contents,
204			**kwargs)
205
206
207			def run_global_bear(message_queue,
208			timeout,
209			global_bear_instance,
210			dependency_results):
211			"""
212			Runs an instance of a global bear. Checks if bear_instance is of type
213			GlobalBear and then passes it to the run_bear to execute.
214
215			:param message_queue: A queue that contains messages of type
216			errors/warnings/debug statements to be printed
217			in the Log.
218			:param timeout: The queue blocks at most timeout seconds for a
219			free slot to execute the put operation on.
220			After the timeout it returns queue Full
221			exception.
222			:param global_bear_instance: Instance of GlobalBear to run.
223			:param dependency_results: The results of all the bears on which the
224			instance of the passed bear to be run depends
225			on.
226			:return: Returns a list of results generated by the
227			passed bear_instance.
228			"""
229			if (not isinstance(global_bear_instance, GlobalBear)
230			or global_bear_instance.kind() != BEAR_KIND.GLOBAL):
231			send_msg(message_queue,
232			timeout,
233			LOG_LEVEL.WARNING,
234			"A given global bear ({}) is not valid. Leaving it "
235			"out..."
236			.format(global_bear_instance.__class__.__name__),
237			Constants.THIS_IS_A_BUG)
238
239			return None
240
241			kwargs = {"dependency_results": dependency_results}
242			return run_bear(message_queue,
243			timeout,
244			global_bear_instance,
245			**kwargs)
246
247
248			def run_local_bears_on_file(message_queue,
249			timeout,
250			local_bear_list,
251			local_result_dict,
252			control_queue,
253			filename,
254			file_contents):
255			"""
256			This method runs a list of local bears on one file.
257
258			:param message_queue: A queue that contains messages of type
259			errors/warnings/debug statements to be printed
260			in the Log.
261			:param timeout: The queue blocks at most timeout seconds for a
262			free slot to execute the put operation on. After
263			the timeout it returns queue Full exception.
264			:param local_bear_list: List of local bears to run on file.
265			:param local_result_dict: A Manager.dict that will be used to store local
266			bear results. A list of all local bear results
267			will be stored with the filename as key.
268			:param control_queue: If any result gets written to the result_dict a
269			tuple containing a CONTROL_ELEMENT (to indicate
270			what kind of event happened) and either a bear
271			name(for global results) or a file name to
272			indicate the result will be put to the queue.
273			:param filename: The name of file on which to run the bears.
274			:param file_contents List that contains contents of the file line by
275			line.
276			"""
277
278			local_result_list = []
279			for bear_instance in local_bear_list:
280			result = run_local_bear(message_queue,
281			timeout,
282			local_result_list,
283			bear_instance,
284			filename,
285			file_contents)
286			if result is not None:
287			local_result_list.extend(result)
288
289			local_result_dict[filename] = local_result_list
290			control_queue.put((CONTROL_ELEMENT.LOCAL, filename))
291
292
293			def get_global_dependency_results(global_result_dict, bear_instance):
294			"""
295			This method gets all the results originating from the dependencies of a
296			bear_instance. Each bear_instance may or may not have dependencies.
297
298			:param global_result_dict: The list of results out of which the dependency
299			results are picked.
300			:return: None if bear has no dependencies, False if
301			dependencies are not met, the dependency dict
302			otherwise.
303			"""
304			try:
305			deps = bear_instance.BEAR_DEPS
306			if not deps:
307			return None
308			except AttributeError:
309			# When this occurs we have an invalid bear and a warning will be
310			# emitted later.
311			return None
312
313			dependency_results = {}
314			for dep in deps:
315			depname = dep.__name__
316			if depname not in global_result_dict:
317			return False
318
319			dependency_results[depname] = global_result_dict[depname]
320
321			return dependency_results
322
323
324			def get_next_global_bear(timeout,
325			global_bear_queue,
326			global_bear_list,
327			global_result_dict):
328			"""
329			Retrieves the next global bear.
330
331			:param timeout: The queue blocks at most timeout seconds for a
332			free slot to execute the put operation on. After
333			the timeout it returns queue Full exception.
334			:param global_bear_queue: queue (read, write) of indexes of global bear
335			instances in the global_bear_list.
336			:param global_bear_list: A list containing all global bears to be
337			executed.
338			:param global_result_dict: A Manager.dict that will be used to store global
339			results. The list of results of one global bear
340			will be stored with the bear name as key.
341			:return: (bear, bearname, dependency_results)
342			"""
343			dependency_results = False
344
345			while dependency_results is False:
346			bear_id = global_bear_queue.get(timeout=timeout)
347			bear = global_bear_list[bear_id]
348
349			dependency_results = (
350			get_global_dependency_results(global_result_dict, bear))
351			if dependency_results is False:
352			global_bear_queue.put(bear_id)
353
354			return bear, dependency_results
355
356
357			def task_done(obj):
358			"""
359			Invokes task_done if the given queue provides this operation. Otherwise
360			passes silently.
361
362			:param obj: Any object.
363			"""
364			if hasattr(obj, "task_done"):
365			obj.task_done()
366
367
368			def run_local_bears(filename_queue,
369			message_queue,
370			timeout,
371			file_dict,
372			local_bear_list,
373			local_result_dict,
374			control_queue):
375			"""
376			Run local bears on all the files given.
377
378			:param filename_queue: queue (read) of file names to check with
379			local bears.
380			:param message_queue: A queue that contains messages of type
381			errors/warnings/debug statements to be printed
382			in the Log.
383			:param timeout: The queue blocks at most timeout seconds for a
384			free slot to execute the put operation on. After
385			the timeout it returns queue Full exception.
386			:param file_dict: Dictionary that contains fileproxy objects.
387			:param local_bear_list: List of local bears to run.
388			:param local_result_dict: A Manager.dict that will be used to store local
389			bear results. A list of all local bear results
390			will be stored with the filename as key.
391			:param control_queue: If any result gets written to the result_dict a
392			tuple containing a CONTROL_ELEMENT (to indicate
393			what kind of event happened) and either a bear
394			name(for global results) or a file name to
395			indicate the result will be put to the queue.
396			"""
397			try:
398			while True:
399			filename = filename_queue.get(timeout=timeout)
400
401			run_local_bears_on_file(message_queue,
402			timeout,
403			local_bear_list,
404			local_result_dict,
405			control_queue,
406			filename,
407			file_dict[filename].lines)
408			task_done(filename_queue)
409			except queue.Empty:
410			return
411
412
413			def run_global_bears(message_queue,
414			timeout,
415			global_bear_queue,
416			global_bear_list,
417			global_result_dict,
418			control_queue):
419			"""
420			Run all global bears.
421
422			:param message_queue: A queue that contains messages of type
423			errors/warnings/debug statements to be printed
424			in the Log.
425			:param timeout: The queue blocks at most timeout seconds for a
426			free slot to execute the put operation on. After
427			the timeout it returns queue Full exception.
428			:param global_bear_queue: queue (read, write) of indexes of global bear
429			instances in the global_bear_list.
430			:param global_bear_list: list of global bear instances
431			:param global_result_dict: A Manager.dict that will be used to store global
432			results. The list of results of one global bear
433			will be stored with the bear name as key.
434			:param control_queue: If any result gets written to the result_dict a
435			tuple containing a CONTROL_ELEMENT (to indicate
436			what kind of event happened) and either a bear
437			name(for global results) or a file name to
438			indicate the result will be put to the queue.
439			"""
440			try:
441			while True:
442			bear, dep_results = (
443			get_next_global_bear(timeout,
444			global_bear_queue,
445			global_bear_list,
446			global_result_dict))
447			bearname = bear.__class__.__name__
448			result = run_global_bear(message_queue, timeout, bear, dep_results)
449			if result:
450			global_result_dict[bearname] = result
451			control_queue.put((CONTROL_ELEMENT.GLOBAL, bearname))
452			else:
453			global_result_dict[bearname] = None
454			task_done(global_bear_queue)
455			except queue.Empty:
456			return
457
458
459			def run(file_name_queue,
460			local_bear_list,
461			global_bear_list,
462			global_bear_queue,
463			file_dict,
464			local_result_dict,
465			global_result_dict,
466			message_queue,
467			control_queue,
468			timeout=0):
469			"""
470			This is the method that is actually runs by processes.
471
472			If parameters type is 'queue (read)' this means it has to implement the
473			get(timeout=TIMEOUT) method and it shall raise queue.Empty if the queue
474			is empty up until the end of the timeout. If the queue has the
475			(optional!) task_done() attribute, the run method will call it after
476			processing each item.
477
478			If parameters type is 'queue (write)' it shall implement the
479			put(object, timeout=TIMEOUT) method.
480
481			If the queues raise any exception not specified here the user will get
482			an 'unknown error' message. So beware of that.
483
484			:param file_name_queue: queue (read) of file names to check with local
485			bears. Each invocation of the run method needs
486			one such queue which it checks with all the
487			local bears. The queue could be empty.
488			(Repeat until queue empty.)
489			:param local_bear_list: List of local bear instances.
490			:param global_bear_list: List of global bear instances.
491			:param global_bear_queue: queue (read, write) of indexes of global bear
492			instances in the global_bear_list.
493			:param file_dict: dict of all files as {filename:file}, file as in
494			the fileproxy object.
495			:param local_result_dict: A Manager.dict that will be used to store local
496			results. A list of all local results.
497			will be stored with the filename as key.
498			:param global_result_dict: A Manager.dict that will be used to store global
499			results. The list of results of one global bear
500			will be stored with the bear name as key.
501			:param message_queue: queue (write) for debug/warning/error
502			messages (type LogMessage)
503			:param control_queue: queue (write). If any result gets written to the
504			result_dict a tuple containing a CONTROL_ELEMENT
505			(to indicate what kind of event happened) and
506			either a bear name (for global results) or a
507			file name to indicate the result will be put to
508			the queue. If the run method finished all its
509			local bears it will put
510			(CONTROL_ELEMENT.LOCAL_FINISHED, None) to the
511			queue, if it finished all global ones,
512			(CONTROL_ELEMENT.GLOBAL_FINISHED, None) will
513			be put there.
514			:param timeout: The queue blocks at most timeout seconds for a
515			free slot to execute the put operation on. After
516			the timeout it returns queue Full exception.
517			"""
518			try:
519			run_local_bears(file_name_queue,
520			message_queue,
521			timeout,
522			file_dict,
523			local_bear_list,
524			local_result_dict,
525			control_queue)
526			control_queue.put((CONTROL_ELEMENT.LOCAL_FINISHED, None))
527
528			run_global_bears(message_queue,
529			timeout,
530			global_bear_queue,
531			global_bear_list,
532			global_result_dict,
533			control_queue)
534			control_queue.put((CONTROL_ELEMENT.GLOBAL_FINISHED, None))
535			except (OSError, KeyboardInterrupt): # pragma: no cover
536			pass
537

coala-analyzer / coala

Pull Request — master (#2609)

run_local_bears_on_file() B

Complexity

Size

Duplication

Importance

Duplication Side-by-Side

Filter issues like