reports.equipmentoutput   F
last analyzed

Complexity

Total Complexity 95

Size/Duplication

Total Lines 473
Duplicated Lines 97.89 %

Importance

Changes 0
Metric Value
eloc 338
dl 463
loc 473
rs 2
c 0
b 0
f 0
wmc 95

3 Methods

Rating   Name   Duplication   Size   Complexity  
A Reporting.on_options() 3 3 1
F Reporting.on_get() 442 442 93
A Reporting.__init__() 3 3 1

How to fix   Duplicated Code    Complexity   

Duplicated Code

Duplicate code is one of the most pungent code smells. A rule that is often used is to re-structure code once it is duplicated in three or more places.

Common duplication problems, and corresponding solutions are:

Complexity

 Tip:   Before tackling complexity, make sure that you eliminate any duplication first. This often can reduce the size of classes significantly.

Complex classes like reports.equipmentoutput often do a lot of different things. To break such a class down, we need to identify a cohesive component within that class. A common approach to find such a component is to look for fields/methods that share the same prefixes, or suffixes.

Once you have determined the fields that belong together, you can apply the Extract Class refactoring. If the component makes sense as a sub-class, Extract Subclass is also a candidate, and is often faster.

1
import falcon
2
import simplejson as json
3
import mysql.connector
4
import config
5
from datetime import datetime, timedelta, timezone
6
from core import utilities
7
from decimal import Decimal
8
9
10 View Code Duplication
class Reporting:
0 ignored issues
show
Duplication introduced by
This code seems to be duplicated in your project.
Loading history...
11
    @staticmethod
12
    def __init__():
13
        pass
14
15
    @staticmethod
16
    def on_options(req, resp):
17
        resp.status = falcon.HTTP_200
18
19
    ####################################################################################################################
20
    # PROCEDURES
21
    # Step 1: valid parameters
22
    # Step 2: query the equipment
23
    # Step 3: query energy categories
24
    # Step 4: query associated points
25
    # Step 5: query base period energy output
26
    # Step 6: query reporting period energy output
27
    # Step 7: query tariff data
28
    # Step 8: query associated points data
29
    # Step 9: construct the report
30
    ####################################################################################################################
31
    @staticmethod
32
    def on_get(req, resp):
33
        print(req.params)
34
        equipment_id = req.params.get('equipmentid')
35
        period_type = req.params.get('periodtype')
36
        base_start_datetime_local = req.params.get('baseperiodstartdatetime')
37
        base_end_datetime_local = req.params.get('baseperiodenddatetime')
38
        reporting_start_datetime_local = req.params.get('reportingperiodstartdatetime')
39
        reporting_end_datetime_local = req.params.get('reportingperiodenddatetime')
40
41
        ################################################################################################################
42
        # Step 1: valid parameters
43
        ################################################################################################################
44
        if equipment_id is None:
45
            raise falcon.HTTPError(falcon.HTTP_400, title='API.BAD_REQUEST', description='API.INVALID_EQUIPMENT_ID')
46
        else:
47
            equipment_id = str.strip(equipment_id)
48
            if not equipment_id.isdigit() or int(equipment_id) <= 0:
49
                raise falcon.HTTPError(falcon.HTTP_400, title='API.BAD_REQUEST', description='API.INVALID_EQUIPMENT_ID')
50
51
        if period_type is None:
52
            raise falcon.HTTPError(falcon.HTTP_400, title='API.BAD_REQUEST', description='API.INVALID_PERIOD_TYPE')
53
        else:
54
            period_type = str.strip(period_type)
55
            if period_type not in ['hourly', 'daily', 'monthly', 'yearly']:
56
                raise falcon.HTTPError(falcon.HTTP_400, title='API.BAD_REQUEST', description='API.INVALID_PERIOD_TYPE')
57
58
        timezone_offset = int(config.utc_offset[1:3]) * 60 + int(config.utc_offset[4:6])
59
        if config.utc_offset[0] == '-':
60
            timezone_offset = -timezone_offset
61
62
        base_start_datetime_utc = None
63
        if base_start_datetime_local is not None and len(str.strip(base_start_datetime_local)) > 0:
64
            base_start_datetime_local = str.strip(base_start_datetime_local)
65
            try:
66
                base_start_datetime_utc = datetime.strptime(base_start_datetime_local,
67
                                                            '%Y-%m-%dT%H:%M:%S').replace(tzinfo=timezone.utc) - \
68
                    timedelta(minutes=timezone_offset)
69
            except ValueError:
70
                raise falcon.HTTPError(falcon.HTTP_400, title='API.BAD_REQUEST',
71
                                       description="API.INVALID_BASE_PERIOD_START_DATETIME")
72
73
        base_end_datetime_utc = None
74
        if base_end_datetime_local is not None and len(str.strip(base_end_datetime_local)) > 0:
75
            base_end_datetime_local = str.strip(base_end_datetime_local)
76
            try:
77
                base_end_datetime_utc = datetime.strptime(base_end_datetime_local,
78
                                                          '%Y-%m-%dT%H:%M:%S').replace(tzinfo=timezone.utc) - \
79
                    timedelta(minutes=timezone_offset)
80
            except ValueError:
81
                raise falcon.HTTPError(falcon.HTTP_400, title='API.BAD_REQUEST',
82
                                       description="API.INVALID_BASE_PERIOD_END_DATETIME")
83
84
        if base_start_datetime_utc is not None and base_end_datetime_utc is not None and \
85
                base_start_datetime_utc >= base_end_datetime_utc:
86
            raise falcon.HTTPError(falcon.HTTP_400, title='API.BAD_REQUEST',
87
                                   description='API.INVALID_BASE_PERIOD_END_DATETIME')
88
89
        if reporting_start_datetime_local is None:
90
            raise falcon.HTTPError(falcon.HTTP_400, title='API.BAD_REQUEST',
91
                                   description="API.INVALID_REPORTING_PERIOD_START_DATETIME")
92
        else:
93
            reporting_start_datetime_local = str.strip(reporting_start_datetime_local)
94
            try:
95
                reporting_start_datetime_utc = datetime.strptime(reporting_start_datetime_local,
96
                                                                 '%Y-%m-%dT%H:%M:%S').replace(tzinfo=timezone.utc) - \
97
                    timedelta(minutes=timezone_offset)
98
            except ValueError:
99
                raise falcon.HTTPError(falcon.HTTP_400, title='API.BAD_REQUEST',
100
                                       description="API.INVALID_REPORTING_PERIOD_START_DATETIME")
101
102
        if reporting_end_datetime_local is None:
103
            raise falcon.HTTPError(falcon.HTTP_400, title='API.BAD_REQUEST',
104
                                   description="API.INVALID_REPORTING_PERIOD_END_DATETIME")
105
        else:
106
            reporting_end_datetime_local = str.strip(reporting_end_datetime_local)
107
            try:
108
                reporting_end_datetime_utc = datetime.strptime(reporting_end_datetime_local,
109
                                                               '%Y-%m-%dT%H:%M:%S').replace(tzinfo=timezone.utc) - \
110
                    timedelta(minutes=timezone_offset)
111
            except ValueError:
112
                raise falcon.HTTPError(falcon.HTTP_400, title='API.BAD_REQUEST',
113
                                       description="API.INVALID_REPORTING_PERIOD_END_DATETIME")
114
115
        if reporting_start_datetime_utc >= reporting_end_datetime_utc:
116
            raise falcon.HTTPError(falcon.HTTP_400, title='API.BAD_REQUEST',
117
                                   description='API.INVALID_REPORTING_PERIOD_END_DATETIME')
118
119
        ################################################################################################################
120
        # Step 2: query the equipment
121
        ################################################################################################################
122
        cnx_system = mysql.connector.connect(**config.myems_system_db)
123
        cursor_system = cnx_system.cursor()
124
125
        cnx_energy = mysql.connector.connect(**config.myems_energy_db)
126
        cursor_energy = cnx_energy.cursor()
127
128
        cnx_historical = mysql.connector.connect(**config.myems_historical_db)
129
        cursor_historical = cnx_historical.cursor()
130
131
        cursor_system.execute(" SELECT id, name, cost_center_id "
132
                              " FROM tbl_equipments "
133
                              " WHERE id = %s ", (equipment_id,))
134
        row_equipment = cursor_system.fetchone()
135
        if row_equipment is None:
136
            if cursor_system:
137
                cursor_system.close()
138
            if cnx_system:
139
                cnx_system.disconnect()
140
141
            if cursor_energy:
142
                cursor_energy.close()
143
            if cnx_energy:
144
                cnx_energy.disconnect()
145
146
            if cnx_historical:
147
                cnx_historical.close()
148
            if cursor_historical:
149
                cursor_historical.disconnect()
150
            raise falcon.HTTPError(falcon.HTTP_404, title='API.NOT_FOUND', description='API.EQUIPMENT_NOT_FOUND')
151
152
        equipment = dict()
153
        equipment['id'] = row_equipment[0]
154
        equipment['name'] = row_equipment[1]
155
        equipment['cost_center_id'] = row_equipment[2]
156
157
        ################################################################################################################
158
        # Step 3: query energy categories
159
        ################################################################################################################
160
        energy_category_set = set()
161
        # query energy categories in base period
162
        cursor_energy.execute(" SELECT DISTINCT(energy_category_id) "
163
                              " FROM tbl_equipment_output_category_hourly "
164
                              " WHERE equipment_id = %s "
165
                              "     AND start_datetime_utc >= %s "
166
                              "     AND start_datetime_utc < %s ",
167
                              (equipment['id'], base_start_datetime_utc, base_end_datetime_utc))
168
        rows_energy_categories = cursor_energy.fetchall()
169
        if rows_energy_categories is not None or len(rows_energy_categories) > 0:
170
            for row_energy_category in rows_energy_categories:
171
                energy_category_set.add(row_energy_category[0])
172
173
        # query energy categories in reporting period
174
        cursor_energy.execute(" SELECT DISTINCT(energy_category_id) "
175
                              " FROM tbl_equipment_output_category_hourly "
176
                              " WHERE equipment_id = %s "
177
                              "     AND start_datetime_utc >= %s "
178
                              "     AND start_datetime_utc < %s ",
179
                              (equipment['id'], reporting_start_datetime_utc, reporting_end_datetime_utc))
180
        rows_energy_categories = cursor_energy.fetchall()
181
        if rows_energy_categories is not None or len(rows_energy_categories) > 0:
182
            for row_energy_category in rows_energy_categories:
183
                energy_category_set.add(row_energy_category[0])
184
185
        # query all energy categories in base period and reporting period
186
        cursor_system.execute(" SELECT id, name, unit_of_measure, kgce, kgco2e "
187
                              " FROM tbl_energy_categories "
188
                              " ORDER BY id ", )
189
        rows_energy_categories = cursor_system.fetchall()
190
        if rows_energy_categories is None or len(rows_energy_categories) == 0:
191
            if cursor_system:
192
                cursor_system.close()
193
            if cnx_system:
194
                cnx_system.disconnect()
195
196
            if cursor_energy:
197
                cursor_energy.close()
198
            if cnx_energy:
199
                cnx_energy.disconnect()
200
201
            if cnx_historical:
202
                cnx_historical.close()
203
            if cursor_historical:
204
                cursor_historical.disconnect()
205
            raise falcon.HTTPError(falcon.HTTP_404,
206
                                   title='API.NOT_FOUND',
207
                                   description='API.ENERGY_CATEGORY_NOT_FOUND')
208
        energy_category_dict = dict()
209
        for row_energy_category in rows_energy_categories:
210
            if row_energy_category[0] in energy_category_set:
211
                energy_category_dict[row_energy_category[0]] = {"name": row_energy_category[1],
212
                                                                "unit_of_measure": row_energy_category[2],
213
                                                                "kgce": row_energy_category[3],
214
                                                                "kgco2e": row_energy_category[4]}
215
216
        ################################################################################################################
217
        # Step 4: query associated points
218
        ################################################################################################################
219
        point_list = list()
220
        cursor_system.execute(" SELECT p.id, p.name, p.units, p.object_type  "
221
                              " FROM tbl_equipments e, tbl_equipments_parameters ep, tbl_points p "
222
                              " WHERE e.id = %s AND e.id = ep.equipment_id AND ep.parameter_type = 'point' "
223
                              "       AND ep.point_id = p.id "
224
                              " ORDER BY p.id ", (equipment['id'],))
225
        rows_points = cursor_system.fetchall()
226
        if rows_points is not None and len(rows_points) > 0:
227
            for row in rows_points:
228
                point_list.append({"id": row[0], "name": row[1], "units": row[2], "object_type": row[3]})
229
230
        ################################################################################################################
231
        # Step 5: query base period energy output
232
        ################################################################################################################
233
        base = dict()
234
        if energy_category_set is not None and len(energy_category_set) > 0:
235
            for energy_category_id in energy_category_set:
236
                base[energy_category_id] = dict()
237
                base[energy_category_id]['timestamps'] = list()
238
                base[energy_category_id]['values'] = list()
239
                base[energy_category_id]['subtotal'] = Decimal(0.0)
240
241
                cursor_energy.execute(" SELECT start_datetime_utc, actual_value "
242
                                      " FROM tbl_equipment_output_category_hourly "
243
                                      " WHERE equipment_id = %s "
244
                                      "     AND energy_category_id = %s "
245
                                      "     AND start_datetime_utc >= %s "
246
                                      "     AND start_datetime_utc < %s "
247
                                      " ORDER BY start_datetime_utc ",
248
                                      (equipment['id'],
249
                                       energy_category_id,
250
                                       base_start_datetime_utc,
251
                                       base_end_datetime_utc))
252
                rows_equipment_hourly = cursor_energy.fetchall()
253
254
                rows_equipment_periodically = utilities.aggregate_hourly_data_by_period(rows_equipment_hourly,
255
                                                                                        base_start_datetime_utc,
256
                                                                                        base_end_datetime_utc,
257
                                                                                        period_type)
258
                for row_equipment_periodically in rows_equipment_periodically:
259
                    current_datetime_local = row_equipment_periodically[0].replace(tzinfo=timezone.utc) + \
260
                                             timedelta(minutes=timezone_offset)
261
                    if period_type == 'hourly':
262
                        current_datetime = current_datetime_local.strftime('%Y-%m-%dT%H:%M:%S')
263
                    elif period_type == 'daily':
264
                        current_datetime = current_datetime_local.strftime('%Y-%m-%d')
265
                    elif period_type == 'monthly':
266
                        current_datetime = current_datetime_local.strftime('%Y-%m')
267
                    elif period_type == 'yearly':
268
                        current_datetime = current_datetime_local.strftime('%Y')
269
270
                    actual_value = Decimal(0.0) if row_equipment_periodically[1] is None \
271
                        else row_equipment_periodically[1]
272
                    base[energy_category_id]['timestamps'].append(current_datetime)
0 ignored issues
show
introduced by
The variable current_datetime does not seem to be defined for all execution paths.
Loading history...
273
                    base[energy_category_id]['values'].append(actual_value)
274
                    base[energy_category_id]['subtotal'] += actual_value
275
276
        ################################################################################################################
277
        # Step 8: query reporting period energy output
278
        ################################################################################################################
279
        reporting = dict()
280
        if energy_category_set is not None and len(energy_category_set) > 0:
281
            for energy_category_id in energy_category_set:
282
                reporting[energy_category_id] = dict()
283
                reporting[energy_category_id]['timestamps'] = list()
284
                reporting[energy_category_id]['values'] = list()
285
                reporting[energy_category_id]['subtotal'] = Decimal(0.0)
286
287
                cursor_energy.execute(" SELECT start_datetime_utc, actual_value "
288
                                      " FROM tbl_equipment_output_category_hourly "
289
                                      " WHERE equipment_id = %s "
290
                                      "     AND energy_category_id = %s "
291
                                      "     AND start_datetime_utc >= %s "
292
                                      "     AND start_datetime_utc < %s "
293
                                      " ORDER BY start_datetime_utc ",
294
                                      (equipment['id'],
295
                                       energy_category_id,
296
                                       reporting_start_datetime_utc,
297
                                       reporting_end_datetime_utc))
298
                rows_equipment_hourly = cursor_energy.fetchall()
299
300
                rows_equipment_periodically = utilities.aggregate_hourly_data_by_period(rows_equipment_hourly,
301
                                                                                        reporting_start_datetime_utc,
302
                                                                                        reporting_end_datetime_utc,
303
                                                                                        period_type)
304
                for row_equipment_periodically in rows_equipment_periodically:
305
                    current_datetime_local = row_equipment_periodically[0].replace(tzinfo=timezone.utc) + \
306
                                             timedelta(minutes=timezone_offset)
307
                    if period_type == 'hourly':
308
                        current_datetime = current_datetime_local.strftime('%Y-%m-%dT%H:%M:%S')
309
                    elif period_type == 'daily':
310
                        current_datetime = current_datetime_local.strftime('%Y-%m-%d')
311
                    elif period_type == 'monthly':
312
                        current_datetime = current_datetime_local.strftime('%Y-%m')
313
                    elif period_type == 'yearly':
314
                        current_datetime = current_datetime_local.strftime('%Y')
315
316
                    actual_value = Decimal(0.0) if row_equipment_periodically[1] is None \
317
                        else row_equipment_periodically[1]
318
                    reporting[energy_category_id]['timestamps'].append(current_datetime)
319
                    reporting[energy_category_id]['values'].append(actual_value)
320
                    reporting[energy_category_id]['subtotal'] += actual_value
321
322
        ################################################################################################################
323
        # Step 9: query tariff data
324
        ################################################################################################################
325
        parameters_data = dict()
326
        parameters_data['names'] = list()
327
        parameters_data['timestamps'] = list()
328
        parameters_data['values'] = list()
329
        if energy_category_set is not None and len(energy_category_set) > 0:
330
            for energy_category_id in energy_category_set:
331
                energy_category_tariff_dict = utilities.get_energy_category_tariffs(equipment['cost_center_id'],
332
                                                                                    energy_category_id,
333
                                                                                    reporting_start_datetime_utc,
334
                                                                                    reporting_end_datetime_utc)
335
                tariff_timestamp_list = list()
336
                tariff_value_list = list()
337
                for k, v in energy_category_tariff_dict.items():
338
                    # convert k from utc to local
339
                    k = k + timedelta(minutes=timezone_offset)
340
                    tariff_timestamp_list.append(k.isoformat()[0:19][0:19])
341
                    tariff_value_list.append(v)
342
343
                parameters_data['names'].append('TARIFF-' + energy_category_dict[energy_category_id]['name'])
344
                parameters_data['timestamps'].append(tariff_timestamp_list)
345
                parameters_data['values'].append(tariff_value_list)
346
347
        ################################################################################################################
348
        # Step 10: query associated points data
349
        ################################################################################################################
350
        for point in point_list:
351
            point_values = []
352
            point_timestamps = []
353
            if point['object_type'] == 'ANALOG_VALUE':
354
                query = (" SELECT utc_date_time, actual_value "
355
                         " FROM tbl_analog_value "
356
                         " WHERE point_id = %s "
357
                         "       AND utc_date_time BETWEEN %s AND %s "
358
                         " ORDER BY utc_date_time ")
359
                cursor_historical.execute(query, (point['id'],
360
                                                  reporting_start_datetime_utc,
361
                                                  reporting_end_datetime_utc))
362
                rows = cursor_historical.fetchall()
363
364
                if rows is not None and len(rows) > 0:
365
                    for row in rows:
366
                        current_datetime_local = row[0].replace(tzinfo=timezone.utc) + \
367
                                                 timedelta(minutes=timezone_offset)
368
                        current_datetime = current_datetime_local.strftime('%Y-%m-%dT%H:%M:%S')
369
                        point_timestamps.append(current_datetime)
370
                        point_values.append(row[1])
371
372
            elif point['object_type'] == 'ENERGY_VALUE':
373
                query = (" SELECT utc_date_time, actual_value "
374
                         " FROM tbl_energy_value "
375
                         " WHERE point_id = %s "
376
                         "       AND utc_date_time BETWEEN %s AND %s "
377
                         " ORDER BY utc_date_time ")
378
                cursor_historical.execute(query, (point['id'],
379
                                                  reporting_start_datetime_utc,
380
                                                  reporting_end_datetime_utc))
381
                rows = cursor_historical.fetchall()
382
383
                if rows is not None and len(rows) > 0:
384
                    for row in rows:
385
                        current_datetime_local = row[0].replace(tzinfo=timezone.utc) + \
386
                                                 timedelta(minutes=timezone_offset)
387
                        current_datetime = current_datetime_local.strftime('%Y-%m-%dT%H:%M:%S')
388
                        point_timestamps.append(current_datetime)
389
                        point_values.append(row[1])
390
            elif point['object_type'] == 'DIGITAL_VALUE':
391
                query = (" SELECT utc_date_time, actual_value "
392
                         " FROM tbl_digital_value "
393
                         " WHERE point_id = %s "
394
                         "       AND utc_date_time BETWEEN %s AND %s ")
395
                cursor_historical.execute(query, (point['id'],
396
                                                  reporting_start_datetime_utc,
397
                                                  reporting_end_datetime_utc))
398
                rows = cursor_historical.fetchall()
399
400
                if rows is not None and len(rows) > 0:
401
                    for row in rows:
402
                        current_datetime_local = row[0].replace(tzinfo=timezone.utc) + \
403
                                                 timedelta(minutes=timezone_offset)
404
                        current_datetime = current_datetime_local.strftime('%Y-%m-%dT%H:%M:%S')
405
                        point_timestamps.append(current_datetime)
406
                        point_values.append(row[1])
407
408
            parameters_data['names'].append(point['name'] + ' (' + point['units'] + ')')
409
            parameters_data['timestamps'].append(point_timestamps)
410
            parameters_data['values'].append(point_values)
411
412
        ################################################################################################################
413
        # Step 12: construct the report
414
        ################################################################################################################
415
        if cursor_system:
416
            cursor_system.close()
417
        if cnx_system:
418
            cnx_system.disconnect()
419
420
        if cursor_energy:
421
            cursor_energy.close()
422
        if cnx_energy:
423
            cnx_energy.disconnect()
424
425
        result = dict()
426
427
        result['equipment'] = dict()
428
        result['equipment']['name'] = equipment['name']
429
430
        result['base_period'] = dict()
431
        result['base_period']['names'] = list()
432
        result['base_period']['units'] = list()
433
        result['base_period']['timestamps'] = list()
434
        result['base_period']['values'] = list()
435
        result['base_period']['subtotals'] = list()
436
        if energy_category_set is not None and len(energy_category_set) > 0:
437
            for energy_category_id in energy_category_set:
438
                result['base_period']['names'].append(energy_category_dict[energy_category_id]['name'])
439
                result['base_period']['units'].append(energy_category_dict[energy_category_id]['unit_of_measure'])
440
                result['base_period']['timestamps'].append(base[energy_category_id]['timestamps'])
441
                result['base_period']['values'].append(base[energy_category_id]['values'])
442
                result['base_period']['subtotals'].append(base[energy_category_id]['subtotal'])
443
444
        result['reporting_period'] = dict()
445
        result['reporting_period']['names'] = list()
446
        result['reporting_period']['energy_category_ids'] = list()
447
        result['reporting_period']['units'] = list()
448
        result['reporting_period']['timestamps'] = list()
449
        result['reporting_period']['values'] = list()
450
        result['reporting_period']['subtotals'] = list()
451
        result['reporting_period']['increment_rates'] = list()
452
453
        if energy_category_set is not None and len(energy_category_set) > 0:
454
            for energy_category_id in energy_category_set:
455
                result['reporting_period']['names'].append(energy_category_dict[energy_category_id]['name'])
456
                result['reporting_period']['energy_category_ids'].append(energy_category_id)
457
                result['reporting_period']['units'].append(energy_category_dict[energy_category_id]['unit_of_measure'])
458
                result['reporting_period']['timestamps'].append(reporting[energy_category_id]['timestamps'])
459
                result['reporting_period']['values'].append(reporting[energy_category_id]['values'])
460
                result['reporting_period']['subtotals'].append(reporting[energy_category_id]['subtotal'])
461
                result['reporting_period']['increment_rates'].append(
462
                    (reporting[energy_category_id]['subtotal'] - base[energy_category_id]['subtotal']) /
463
                    base[energy_category_id]['subtotal']
464
                    if base[energy_category_id]['subtotal'] > 0.0 else None)
465
466
        result['parameters'] = {
467
            "names": parameters_data['names'],
468
            "timestamps": parameters_data['timestamps'],
469
            "values": parameters_data['values']
470
        }
471
472
        resp.body = json.dumps(result)
473