osm_poi_matchmaker.dataproviders.hu_pepco.hu_pepco.process() - Code Metrics - KAMI911/osm_poi_matchmaker - Measure and Improve Code Quality continuously with Scrutinizer

hu_pepco.process() C
last analyzed 2023-01-03 16:24 UTC

↳ Parent: osm_poi_matchmaker.dataproviders.hu_pepco

Complexity

Conditions

Size

Total Lines	47
Code Lines	35

Duplication

Lines	0
Ratio	0 %

Importance

Changes

Metric	Value
cc	10
eloc	35
nop	1
dl	0
loc	47
rs	5.9999
c	0
b	0
f	0

How to fix Complexity

# -*- coding: utf-8 -*-


try:
    import logging
    import sys
    import os
    import json
    from osm_poi_matchmaker.libs.soup import save_downloaded_soup
    from osm_poi_matchmaker.libs.address import extract_street_housenumber_better_2, clean_city, clean_phone_to_str

    from osm_poi_matchmaker.libs.geo import check_hu_boundary
    from osm_poi_matchmaker.libs.osm import query_osm_city_name
    from osm_poi_matchmaker.libs.osm_tag_sets import POS_HU_GEN, PAY_CASH
    from osm_poi_matchmaker.utils.data_provider import DataProvider
    from osm_poi_matchmaker.utils.enums import FileType
except ImportError as err:
    logging.error('Error %s import module: %s', __name__, err)
    logging.exception('Exception occurred')

    sys.exit(128)


class hu_pepco(DataProvider):


    def constains(self):
        self.link = 'https://pepco.hu/uzleteink/uzletkereso/?type=1002&tx_pepco_mapplugin[action]=view&tx_pepco_mapplugin[controller]=Map&tx_pepco_mapplugin[loadall]=true'

        self.tags = {'shop': 'clothes', 'brand': 'Pepco', 'brand:wikidata': 'Q11815580',
                     'brand:wikipedia': 'pl:Pepco', 'contact:facebook': 'https://www.facebook.com/pepcohu/',

                     'contact:website': 'https://pepco.hu/',
                     'contact:linkedin': 'https://www.linkedin.com/company/pepco-poland',
                     'contact:phone': '+36 1 701 0424', 'contact:email': '[email protected]',
                     'operator': 'Pepkor Hungary Kft.', 'operator:addr': '1138 Budapest, Váci út 187.'}

        self.filetype = FileType.json
        self.filename = '{}.{}'.format(
            self.__class__.__name__, self.filetype.name)

    def types(self):
        hupepcoclo = self.tags.copy()
        hupepcoclo.update(POS_HU_GEN)
        hupepcoclo.update(PAY_CASH)
        self.__types = [
            {'poi_code': 'hupepcoclo', 'poi_name': 'Pepco', 'poi_type': 'clothes',
             'poi_tags': hupepcoclo, 'poi_url_base': 'https://pepco.hu', 'poi_search_name': 'pepco',
             'osm_search_distance_perfect': 2000, 'osm_search_distance_safe': 200,
             'osm_search_distance_unsafe': 5},
        ]
        return self.__types

    def process(self):
        try:
            soup = save_downloaded_soup('{}'.format(self.link), os.path.join(self.download_cache, self.filename),

                                        self.filetype)
            if soup is not None:
                text = json.loads(soup)
                for poi_data in text['data']:
                    '''
                    The Pepco dataset contains all European data. Since the program cannot handle POIs outside Hungary (so far)

                    this will limit only for Hungarian POIs
                    In fact this depends on OSM extract but currently we use only Hungarian OSM extract

                    Select only Hungarian POIs
                    '''

                    if 'city' in poi_data and (poi_data['city'] == '' or

                                               query_osm_city_name(self.session, poi_data['city']) is None):

                        continue
                    elif 'city' in poi_data:
                        self.data.city = clean_city(poi_data['city'])
                    else:
                        continue
                    self.data.name = 'Pepco'
                    self.data.code = 'hupepcoclo'
                    # Assign: code, postcode, city, name, branch, website, original, street, housenumber, conscriptionnumber, ref, geom

                    self.data.lat, self.data.lon = \
                        check_hu_boundary(
                            poi_data['coordinates']['lat'], poi_data['coordinates']['lng'])
                    self.data.street, self.data.housenumber, self.data.conscriptionnumber = \
                        extract_street_housenumber_better_2(
                            poi_data.get('streetAddress'))
                    self.data.original = poi_data.get('streetAddress')
                    self.data.postcode = poi_data.get('postalCode')
                    # self.data.city = query_osm_city_name_gpd(self.session, self.data.lat, self.data.lon)

                    # Assign opening_hours
                    opening = poi_data['openingHours']
                    for i in range(0, 7):
                        if i in opening:
                            self.data.day_open(i, opening[i]['from'])
                            self.data.day_close(i, opening[i]['to'])
                    # Assign additional informations
                    self.data.phone = clean_phone_to_str(
                        poi_data.get('phoneNumber'))
                    self.data.public_holiday_open = False
                    self.data.add()
        except Exception as e:

            logging.exception('Exception occurred')

            logging.error(logging.error(e))


1			# -- coding: utf-8 --
			0 ignored issues – show introduced 2020-10-11 06:10 UTC by Report Bug Copy Issue Report Missing module docstring Loading history...
2
3			try:
4			import logging
5			import sys
6			import os
7			import json
8			from osm_poi_matchmaker.libs.soup import save_downloaded_soup
9			from osm_poi_matchmaker.libs.address import extract_street_housenumber_better_2, clean_city, clean_phone_to_str
			0 ignored issues – show Coding Style introduced 2020-10-11 06:10 UTC by Report Bug Copy Issue Report This line is too long as per the coding-style (115/100). This check looks for lines that are too long. You can specify the maximum line length. Loading history...
10			from osm_poi_matchmaker.libs.geo import check_hu_boundary
11			from osm_poi_matchmaker.libs.osm import query_osm_city_name
12			from osm_poi_matchmaker.libs.osm_tag_sets import POS_HU_GEN, PAY_CASH
13			from osm_poi_matchmaker.utils.data_provider import DataProvider
14			from osm_poi_matchmaker.utils.enums import FileType
15			except ImportError as err:
16			logging.error('Error %s import module: %s', __name__, err)
17			logging.exception('Exception occurred')
18
19			sys.exit(128)
20
21
22			class hu_pepco(DataProvider):
			0 ignored issues – show Coding Style Naming introduced 2020-10-11 06:10 UTC by Report Bug Copy Issue Report Class name "hu_pepco" doesn't conform to PascalCase naming style ('[^\\W\\da-z][^\\W_]+$' pattern) This check looks for invalid names for a range of different identifiers. You can set regular expressions to which the identifiers must conform if the defaults do not match your requirements. If your project includes a Pylint configuration file, the settings contained in that file take precedence. To find out more about Pylint, please refer to their site. Loading history... introduced 2020-10-11 06:10 UTC by Report Bug Copy Issue Report Missing class docstring Loading history...
23
24			def constains(self):
25			self.link = 'https://pepco.hu/uzleteink/uzletkereso/?type=1002&tx_pepco_mapplugin[action]=view&tx_pepco_mapplugin[controller]=Map&tx_pepco_mapplugin[loadall]=true'
			0 ignored issues – show Coding Style introduced 2020-10-11 06:10 UTC by Report Bug Copy Issue Report This line is too long as per the coding-style (171/100). This check looks for lines that are too long. You can specify the maximum line length. Loading history...
26			self.tags = {'shop': 'clothes', 'brand': 'Pepco', 'brand:wikidata': 'Q11815580',
27			'brand:wikipedia': 'pl:Pepco', 'contact:facebook': 'https://www.facebook.com/pepcohu/',
			0 ignored issues – show Coding Style introduced 2020-10-24 07:32 UTC by Report Bug Copy Issue Report This line is too long as per the coding-style (108/100). This check looks for lines that are too long. You can specify the maximum line length. Loading history...
28			'contact:website': 'https://pepco.hu/',
29			'contact:linkedin': 'https://www.linkedin.com/company/pepco-poland',
30			'contact:phone': '+36 1 701 0424', 'contact:email': '[email protected]',
31			'operator': 'Pepkor Hungary Kft.', 'operator:addr': '1138 Budapest, Váci út 187.'}
			0 ignored issues – show Coding Style introduced 2020-10-24 07:32 UTC by Report Bug Copy Issue Report This line is too long as per the coding-style (103/100). This check looks for lines that are too long. You can specify the maximum line length. Loading history...
32			self.filetype = FileType.json
33			self.filename = '{}.{}'.format(
34			self.__class__.__name__, self.filetype.name)
35
36			def types(self):
37			hupepcoclo = self.tags.copy()
38			hupepcoclo.update(POS_HU_GEN)
39			hupepcoclo.update(PAY_CASH)
40			self.__types = [
41			{'poi_code': 'hupepcoclo', 'poi_name': 'Pepco', 'poi_type': 'clothes',
42			'poi_tags': hupepcoclo, 'poi_url_base': 'https://pepco.hu', 'poi_search_name': 'pepco',
43			'osm_search_distance_perfect': 2000, 'osm_search_distance_safe': 200,
44			'osm_search_distance_unsafe': 5},
45			]
46			return self.__types
47
48			def process(self):
49			try:
50			soup = save_downloaded_soup('{}'.format(self.link), os.path.join(self.download_cache, self.filename),
			0 ignored issues – show Coding Style introduced 2020-10-11 06:10 UTC by Report Bug Copy Issue Report This line is too long as per the coding-style (113/100). This check looks for lines that are too long. You can specify the maximum line length. Loading history...
51			self.filetype)
52			if soup is not None:
53			text = json.loads(soup)
54			for poi_data in text['data']:
55			'''
56			The Pepco dataset contains all European data. Since the program cannot handle POIs outside Hungary (so far)
			0 ignored issues – show Coding Style introduced 2020-10-11 06:10 UTC by Report Bug Copy Issue Report This line is too long as per the coding-style (127/100). This check looks for lines that are too long. You can specify the maximum line length. Loading history...
57			this will limit only for Hungarian POIs
58			In fact this depends on OSM extract but currently we use only Hungarian OSM extract
			0 ignored issues – show Coding Style introduced 2020-10-11 06:10 UTC by Report Bug Copy Issue Report This line is too long as per the coding-style (103/100). This check looks for lines that are too long. You can specify the maximum line length. Loading history...
59			Select only Hungarian POIs
60			'''
			0 ignored issues – show Unused Code introduced 2020-10-11 06:10 UTC by Report Bug Copy Issue Report This string statement has no effect and could be removed. Loading history...
61			if 'city' in poi_data and (poi_data['city'] == '' or
			0 ignored issues – show Unused Code introduced 2020-10-11 06:10 UTC by Report Bug Copy Issue Report Unnecessary "elif" after "continue" Loading history...
62			query_osm_city_name(self.session, poi_data['city']) is None):
			0 ignored issues – show Coding Style introduced 2020-10-19 20:05 UTC by Report Bug Copy Issue Report This line is too long as per the coding-style (108/100). This check looks for lines that are too long. You can specify the maximum line length. Loading history...
63			continue
64			elif 'city' in poi_data:
65			self.data.city = clean_city(poi_data['city'])
66			else:
67			continue
68			self.data.name = 'Pepco'
69			self.data.code = 'hupepcoclo'
70			# Assign: code, postcode, city, name, branch, website, original, street, housenumber, conscriptionnumber, ref, geom
			0 ignored issues – show Coding Style introduced 2020-10-11 06:10 UTC by Report Bug Copy Issue Report This line is too long as per the coding-style (135/100). This check looks for lines that are too long. You can specify the maximum line length. Loading history...
71			self.data.lat, self.data.lon = \
72			check_hu_boundary(
73			poi_data['coordinates']['lat'], poi_data['coordinates']['lng'])
74			self.data.street, self.data.housenumber, self.data.conscriptionnumber = \
75			extract_street_housenumber_better_2(
76			poi_data.get('streetAddress'))
77			self.data.original = poi_data.get('streetAddress')
78			self.data.postcode = poi_data.get('postalCode')
79			# self.data.city = query_osm_city_name_gpd(self.session, self.data.lat, self.data.lon)
			0 ignored issues – show Coding Style introduced 2020-10-11 06:10 UTC by Report Bug Copy Issue Report This line is too long as per the coding-style (106/100). This check looks for lines that are too long. You can specify the maximum line length. Loading history...
80			# Assign opening_hours
81			opening = poi_data['openingHours']
82			for i in range(0, 7):
83			if i in opening:
84			self.data.day_open(i, opening[i]['from'])
85			self.data.day_close(i, opening[i]['to'])
86			# Assign additional informations
87			self.data.phone = clean_phone_to_str(
88			poi_data.get('phoneNumber'))
89			self.data.public_holiday_open = False
90			self.data.add()
91			except Exception as e:
			0 ignored issues – show Best Practice introduced 2020-10-13 09:58 UTC by Report Bug Copy Issue Report Catching very general exceptions such as `Exception` is usually not recommended. Generally, you would want to handle very specific errors in the exception handler. This ensure that you do not hide other types of errors which should be fixed. So, unless you specifically plan to handle any error, consider adding a more specific exception. Loading history... Coding Style Naming introduced 2020-10-13 09:58 UTC by Report Bug Copy Issue Report Variable name "e" doesn't conform to snake_case naming style ('([^\\W\\dA-Z][^\\WA-Z]2,\|_[^\\WA-Z]*\|__[^\\WA-Z\\d_][^\\WA-Z]+__)$' pattern) This check looks for invalid names for a range of different identifiers. You can set regular expressions to which the identifiers must conform if the defaults do not match your requirements. If your project includes a Pylint configuration file, the settings contained in that file take precedence. To find out more about Pylint, please refer to their site. Loading history...
92			logging.exception('Exception occurred')
93
94			logging.error(logging.error(e))
95

KAMI911 / osm_poi_matchmaker

hu_pepco.process() C last analyzed 2023-01-03 16:24 UTC

Complexity

Size

Duplication

Importance

How to fix Complexity

Complexity

Duplication Side-by-Side

Filter issues like

hu_pepco.process() C
last analyzed 2023-01-03 16:24 UTC