osm_poi_matchmaker.dataproviders.hu_benu.hu_benu.process() - Code Metrics - Inspection of "Refactor: convert string type fake dictionaries to..." - KAMI911/osm_poi_matchmaker - Measure and Improve Code Quality continuously with Scrutinizer

Passed

Push — master ( de54f1...6a01a0 )

by KAMI

created 2020-10-26 07:41 UTC

hu_benu.process() C

↳ Parent: osm_poi_matchmaker.dataproviders.hu_benu

Complexity

Conditions

Size

Total Lines	41
Code Lines	37

Duplication

Lines	0
Ratio	0 %

Importance

Changes

Metric	Value
cc	10
eloc	37
nop	1
dl	0
loc	41
rs	5.9999
c	0
b	0
f	0

How to fix Complexity

# -*- coding: utf-8 -*-


try:
    import logging
    import sys
    import os
    import json
    from osm_poi_matchmaker.dao.data_handlers import insert_poi_dataframe

    from osm_poi_matchmaker.libs.soup import save_downloaded_soup
    from osm_poi_matchmaker.libs.address import extract_street_housenumber_better_2, clean_city, clean_phone_to_str, \

        PATTERN_FULL_URL
    from osm_poi_matchmaker.libs.geo import check_hu_boundary
    from osm_poi_matchmaker.libs.osm_tag_sets import POS_HU_GEN, PAY_CASH
    from osm_poi_matchmaker.utils.data_provider import DataProvider
    from osm_poi_matchmaker.utils.enums import FileType
except ImportError as err:
    logging.error('Error %s import module: %s', __name__, err)
    logging.exception('Exception occurred')

    sys.exit(128)


class hu_benu(DataProvider):


    def constains(self):

        self.link = 'https://benu.hu/wordpress-core/wp-admin/admin-ajax.php?action=asl_load_stores&nonce=1900018ba1&load_all=1&layout=1'

        self.tags = {'brand': 'Benu gyógyszertár', 'dispensing': 'yes',
                     'contact:facebook': 'https://www.facebook.com/BENUgyogyszertar',
                     'contact:youtube': 'https://www.youtube.com/channel/UCBLjL10QMtRHdkak0h9exqg',
                     'air_conditioning': 'yes', }
        self.tags.update(POS_HU_GEN)
        self.tags.update(PAY_CASH)
        self.filetype = FileType.json
        self.filename = '{}.{}'.format(
            self.__class__.__name__, self.filetype.name)

    def types(self):
        hubenupha = {'amenity': 'pharmacy'}
        hubenupha.update(self.tags)
        self.__types = [
            {'poi_code': 'hubenupha', 'poi_name': 'Benu gyógyszertár', 'poi_type': 'pharmacy',
             'poi_tags': hubenupha, 'poi_url_base': 'https://benu.hu',
             'poi_search_name': '(benu gyogyszertár|benu)',
             'osm_search_distance_perfect': 2000, 'osm_search_distance_safe': 200,
             'osm_search_distance_unsafe': 20, 'preserve_original_name': True},
        ]
        return self.__types

    def process(self):
        try:
            soup = save_downloaded_soup('{}'.format(self.link), os.path.join(self.download_cache, self.filename),

                                        self.filetype)
            if soup is not None:
                text = json.loads(str(soup))
                for poi_data in text:
                    try:
                        if 'BENU Gyógyszertár' not in poi_data.get('title'):
                            self.data.name = poi_data.get('title').strip()
                            self.data.branch = None
                        else:
                            self.data.name = 'Benu gyógyszertár'
                            self.data.branch = poi_data.get('title').strip()
                        self.data.code = 'hubenupha'
                        if poi_data.get('description') is not None:
                            pu_match = PATTERN_FULL_URL.match(poi_data.get('description'))
                            self.data.website = pu_match.group(0).strip() if pu_match is not None else None

                        else:
                            self.data.website = None
                        self.data.city = clean_city(poi_data.get('city'))
                        self.data.postcode = poi_data.get('postal_code').strip()
                        self.data.lat, self.data.lon = check_hu_boundary(poi_data.get('lat'), poi_data.get('lng'))

                        self.data.street, self.data.housenumber, self.data.conscriptionnumber = extract_street_housenumber_better_2(

                            poi_data.get(('street')))
                        self.data.original = poi_data.get('street')
                        if 'phone' in poi_data and poi_data.get('phone') != '':
                            self.data.phone = clean_phone_to_str(
                                poi_data.get('phone'))
                        else:
                            self.data.phone = None
                        self.data.public_holiday_open = False
                        self.data.add()
                    except Exception as e:

                        logging.error(e)
                        logging.error(poi_data)
                        logging.exception('Exception occurred')

        except Exception as e:

            logging.error(e)
            logging.exception('Exception occurred')


1		# -- coding: utf-8 --
		0 ignored issues – show introduced 2020-10-11 06:10 UTC by Report Bug Copy Issue Report Missing module docstring Loading history...
2
3		try:
4		import logging
5		import sys
6		import os
7		import json
8		from osm_poi_matchmaker.dao.data_handlers import insert_poi_dataframe
		0 ignored issues – show Unused Code introduced 2020-10-11 06:10 UTC by Report Bug Copy Issue Report Unused insert_poi_dataframe imported from osm_poi_matchmaker.dao.data_handlers Loading history...
9		from osm_poi_matchmaker.libs.soup import save_downloaded_soup
10		from osm_poi_matchmaker.libs.address import extract_street_housenumber_better_2, clean_city, clean_phone_to_str, \
		0 ignored issues – show Coding Style introduced 2020-10-11 06:10 UTC by Report Bug Copy Issue Report This line is too long as per the coding-style (118/100). This check looks for lines that are too long. You can specify the maximum line length. Loading history...
11		PATTERN_FULL_URL
12		from osm_poi_matchmaker.libs.geo import check_hu_boundary
13		from osm_poi_matchmaker.libs.osm_tag_sets import POS_HU_GEN, PAY_CASH
14		from osm_poi_matchmaker.utils.data_provider import DataProvider
15		from osm_poi_matchmaker.utils.enums import FileType
16		except ImportError as err:
17		logging.error('Error %s import module: %s', __name__, err)
18		logging.exception('Exception occurred')
19
20		sys.exit(128)
21
22
23		class hu_benu(DataProvider):
		0 ignored issues – show Coding Style Naming introduced 2020-10-11 06:10 UTC by Report Bug Copy Issue Report Class name "hu_benu" doesn't conform to PascalCase naming style ('[^\\W\\da-z][^\\W_]+$' pattern) This check looks for invalid names for a range of different identifiers. You can set regular expressions to which the identifiers must conform if the defaults do not match your requirements. If your project includes a Pylint configuration file, the settings contained in that file take precedence. To find out more about Pylint, please refer to their site. Loading history... introduced 2020-10-11 06:10 UTC by Report Bug Copy Issue Report Missing class docstring Loading history...
24
25	View Code Duplication	def constains(self):
		0 ignored issues – show Duplication introduced 2020-10-24 07:32 UTC by Report Bug Copy Issue Report This code seems to be duplicated in your project. Loading history...
26		self.link = 'https://benu.hu/wordpress-core/wp-admin/admin-ajax.php?action=asl_load_stores&nonce=1900018ba1&load_all=1&layout=1'
		0 ignored issues – show Coding Style introduced 2020-10-11 06:10 UTC by Report Bug Copy Issue Report This line is too long as per the coding-style (136/100). This check looks for lines that are too long. You can specify the maximum line length. Loading history...
27		self.tags = {'brand': 'Benu gyógyszertár', 'dispensing': 'yes',
28		'contact:facebook': 'https://www.facebook.com/BENUgyogyszertar',
29		'contact:youtube': 'https://www.youtube.com/channel/UCBLjL10QMtRHdkak0h9exqg',
30		'air_conditioning': 'yes', }
31		self.tags.update(POS_HU_GEN)
32		self.tags.update(PAY_CASH)
33		self.filetype = FileType.json
34		self.filename = '{}.{}'.format(
35		self.__class__.__name__, self.filetype.name)
36
37		def types(self):
38		hubenupha = {'amenity': 'pharmacy'}
39		hubenupha.update(self.tags)
40		self.__types = [
41		{'poi_code': 'hubenupha', 'poi_name': 'Benu gyógyszertár', 'poi_type': 'pharmacy',
42		'poi_tags': hubenupha, 'poi_url_base': 'https://benu.hu',
43		'poi_search_name': '(benu gyogyszertár\|benu)',
44		'osm_search_distance_perfect': 2000, 'osm_search_distance_safe': 200,
45		'osm_search_distance_unsafe': 20, 'preserve_original_name': True},
46		]
47		return self.__types
48
49		def process(self):
50		try:
51		soup = save_downloaded_soup('{}'.format(self.link), os.path.join(self.download_cache, self.filename),
		0 ignored issues – show Coding Style introduced 2020-10-11 06:10 UTC by Report Bug Copy Issue Report This line is too long as per the coding-style (113/100). This check looks for lines that are too long. You can specify the maximum line length. Loading history...
52		self.filetype)
53		if soup is not None:
54		text = json.loads(str(soup))
55		for poi_data in text:
56		try:
57		if 'BENU Gyógyszertár' not in poi_data.get('title'):
58		self.data.name = poi_data.get('title').strip()
59		self.data.branch = None
60		else:
61		self.data.name = 'Benu gyógyszertár'
62		self.data.branch = poi_data.get('title').strip()
63		self.data.code = 'hubenupha'
64		if poi_data.get('description') is not None:
65		pu_match = PATTERN_FULL_URL.match(poi_data.get('description'))
66		self.data.website = pu_match.group(0).strip() if pu_match is not None else None
		0 ignored issues – show Coding Style introduced 2020-10-11 06:10 UTC by Report Bug Copy Issue Report This line is too long as per the coding-style (107/100). This check looks for lines that are too long. You can specify the maximum line length. Loading history...
67		else:
68		self.data.website = None
69		self.data.city = clean_city(poi_data.get('city'))
70		self.data.postcode = poi_data.get('postal_code').strip()
71		self.data.lat, self.data.lon = check_hu_boundary(poi_data.get('lat'), poi_data.get('lng'))
		0 ignored issues – show Coding Style introduced 2020-10-11 06:10 UTC by Report Bug Copy Issue Report This line is too long as per the coding-style (114/100). This check looks for lines that are too long. You can specify the maximum line length. Loading history...
72		self.data.street, self.data.housenumber, self.data.conscriptionnumber = extract_street_housenumber_better_2(
		0 ignored issues – show Coding Style introduced 2020-10-11 06:10 UTC by Report Bug Copy Issue Report This line is too long as per the coding-style (132/100). This check looks for lines that are too long. You can specify the maximum line length. Loading history...
73		poi_data.get(('street')))
74		self.data.original = poi_data.get('street')
75		if 'phone' in poi_data and poi_data.get('phone') != '':
76		self.data.phone = clean_phone_to_str(
77		poi_data.get('phone'))
78		else:
79		self.data.phone = None
80		self.data.public_holiday_open = False
81		self.data.add()
82		except Exception as e:
		0 ignored issues – show Best Practice introduced 2020-10-11 06:10 UTC by Report Bug Copy Issue Report Catching very general exceptions such as `Exception` is usually not recommended. Generally, you would want to handle very specific errors in the exception handler. This ensure that you do not hide other types of errors which should be fixed. So, unless you specifically plan to handle any error, consider adding a more specific exception. Loading history... Coding Style Naming introduced 2020-10-11 06:10 UTC by Report Bug Copy Issue Report Variable name "e" doesn't conform to snake_case naming style ('([^\\W\\dA-Z][^\\WA-Z]2,\|_[^\\WA-Z]*\|__[^\\WA-Z\\d_][^\\WA-Z]+__)$' pattern) This check looks for invalid names for a range of different identifiers. You can set regular expressions to which the identifiers must conform if the defaults do not match your requirements. If your project includes a Pylint configuration file, the settings contained in that file take precedence. To find out more about Pylint, please refer to their site. Loading history...
83		logging.error(e)
84		logging.error(poi_data)
85		logging.exception('Exception occurred')
86
87		except Exception as e:
		0 ignored issues – show Best Practice introduced 2020-10-13 09:58 UTC by Report Bug Copy Issue Report Catching very general exceptions such as `Exception` is usually not recommended. Generally, you would want to handle very specific errors in the exception handler. This ensure that you do not hide other types of errors which should be fixed. So, unless you specifically plan to handle any error, consider adding a more specific exception. Loading history... Coding Style Naming introduced 2020-10-13 09:58 UTC by Report Bug Copy Issue Report Variable name "e" doesn't conform to snake_case naming style ('([^\\W\\dA-Z][^\\WA-Z]2,\|_[^\\WA-Z]*\|__[^\\WA-Z\\d_][^\\WA-Z]+__)$' pattern) This check looks for invalid names for a range of different identifiers. You can set regular expressions to which the identifiers must conform if the defaults do not match your requirements. If your project includes a Pylint configuration file, the settings contained in that file take precedence. To find out more about Pylint, please refer to their site. Loading history...
88		logging.error(e)
89		logging.exception('Exception occurred')
90

KAMI911 / osm_poi_matchmaker

Push — master ( de54f1...6a01a0 )

hu_benu.process() C

Complexity

Size

Duplication

Importance

How to fix Complexity

Complexity

Duplication Side-by-Side

Filter issues like