Passed
Pull Request — dev (#880)
by
unknown
01:29
created

motorized_individual_travel.model_timeseries   B

Complexity

Total Complexity 44

Size/Duplication

Total Lines 1081
Duplicated Lines 0 %

Importance

Changes 0
Metric Value
wmc 44
eloc 605
dl 0
loc 1081
rs 8.875
c 0
b 0
f 0

11 Functions

Rating   Name   Duplication   Size   Complexity  
B data_preprocessing() 0 94 3
A generate_static_params() 0 40 1
B load_evs_trips() 0 83 4
D generate_load_time_series() 0 212 10
B generate_model_data_bunch() 0 87 3
A generate_model_data_eGon100RE_remaining() 0 7 1
A generate_model_data_eGon2035_remaining() 0 7 1
A load_grid_district_ids() 0 7 2
A delete_model_data_from_db() 0 52 2
A generate_model_data_grid_district() 0 56 2
F write_model_data_to_db() 0 348 15

How to fix   Complexity   

Complexity

Complex classes like motorized_individual_travel.model_timeseries often do a lot of different things. To break such a class down, we need to identify a cohesive component within that class. A common approach to find such a component is to look for fields/methods that share the same prefixes, or suffixes.

Once you have determined the fields that belong together, you can apply the Extract Class refactoring. If the component makes sense as a sub-class, Extract Subclass is also a candidate, and is often faster.

1
"""
2
Generate timeseries for eTraGo and pypsa-eur-sec
3
4
Call order
5
  * generate_model_data_eGon2035() / generate_model_data_eGon100RE()
6
    * generate_model_data()
7
      * generate_model_data_grid_district()
8
        * load_evs_trips()
9
        * data_preprocessing()
10
        * generate_load_time_series()
11
        * write_model_data_to_db()
12
13
Notes
14
-----
15
# TODO REWORK
16
Share of EV with access to private charging infrastructure (`flex_share`) for
17
use cases work and home are not supported by simBEV v0.1.2 and are applied here
18
(after simulation). Applying those fixed shares post-simulation introduces
19
small errors compared to application during simBEV's trip generation.
20
21
Values (cf. `flex_share` in scenario parameters
22
:func:`egon.data.datasets.scenario_parameters.parameters.mobility`) were
23
linearly extrapolated based upon
24
https://nationale-leitstelle.de/wp-content/pdf/broschuere-lis-2025-2030-final.pdf
25
(p.92):
26
* eGon2035: home=0.8, work=1.0
27
* eGon100RE: home=1.0, work=1.0
28
"""
29
30
from collections import Counter
31
from pathlib import Path
32
import datetime as dt
33
import json
34
35
from sqlalchemy.sql import func
36
import numpy as np
37
import pandas as pd
38
39
from egon.data import db
40
from egon.data.datasets.emobility.motorized_individual_travel.db_classes import (
41
    EgonEvMvGridDistrict,
42
    EgonEvPool,
43
    EgonEvTrip,
44
)
45
from egon.data.datasets.emobility.motorized_individual_travel.helpers import (
46
    DATASET_CFG,
47
    MVGD_MIN_COUNT,
48
    WORKING_DIR,
49
    read_simbev_metadata_file,
50
    reduce_mem_usage,
51
)
52
from egon.data.datasets.etrago_setup import (
53
    EgonPfHvBus,
54
    EgonPfHvLink,
55
    EgonPfHvLinkTimeseries,
56
    EgonPfHvLoad,
57
    EgonPfHvLoadTimeseries,
58
    EgonPfHvStore,
59
    EgonPfHvStoreTimeseries,
60
)
61
from egon.data.datasets.mv_grid_districts import MvGridDistricts
62
63
# from egon.data.datasets.scenario_parameters import get_sector_parameters
64
65
66
def data_preprocessing(
67
    scenario_data: pd.DataFrame, ev_data_df: pd.DataFrame
68
) -> pd.DataFrame:
69
    """Filter SimBEV data to match region requirements. Duplicates profiles
70
    if necessary. Pre-calculates necessary parameters for the load time series.
71
72
    Parameters
73
    ----------
74
    scenario_data : pd.Dataframe
75
        EV per grid district
76
    ev_data_df : pd.Dataframe
77
        Trip data
78
79
    Returns
80
    -------
81
    pd.Dataframe
82
        Trip data
83
    """
84
    # get ev data for given profiles
85
    ev_data_df = ev_data_df.loc[
86
        ev_data_df.ev_id.isin(scenario_data.ev_id.unique())
87
    ]
88
89
    # drop faulty data
90
    ev_data_df = ev_data_df.loc[ev_data_df.park_start <= ev_data_df.park_end]
91
92
    # calculate time necessary to fulfill the charging demand and brutto
93
    # charging capacity in MVA
94
    ev_data_df = ev_data_df.assign(
95
        charging_capacity_grid_MW=(
96
            ev_data_df.charging_capacity_grid / 10 ** 3
97
        ),
98
        minimum_charging_time=(
99
            ev_data_df.charging_demand
100
            / ev_data_df.charging_capacity_nominal
101
            * 4
102
        ),
103
        location=ev_data_df.location.str.replace("/", "_"),
104
    )
105
106
    # fix driving events
107
    ev_data_df.minimum_charging_time.fillna(0, inplace=True)
108
109
    # calculate charging capacity for last timestep
110
    (
111
        full_timesteps,
112
        last_timestep_share,
113
    ) = ev_data_df.minimum_charging_time.divmod(1)
114
115
    full_timesteps = full_timesteps.astype(int)
116
117
    ev_data_df = ev_data_df.assign(
118
        full_timesteps=full_timesteps,
119
        last_timestep_share=last_timestep_share,
120
        last_timestep_charging_capacity_grid_MW=(
121
            last_timestep_share * ev_data_df.charging_capacity_grid_MW
122
        ),
123
        charge_end=ev_data_df.park_start + full_timesteps,
124
        last_timestep=ev_data_df.park_start + full_timesteps,
125
    )
126
127
    # Calculate flexible charging capacity:
128
    # only for private charging facilities at home and work
129
    mask_work = (ev_data_df.location == "0_work") & (
130
        ev_data_df.use_case == "work"
131
    )
132
    mask_home = (ev_data_df.location == "6_home") & (
133
        ev_data_df.use_case == "home"
134
    )
135
136
    ev_data_df["flex_charging_capacity_grid_MW"] = 0
137
    ev_data_df.loc[
138
        mask_work | mask_home, "flex_charging_capacity_grid_MW"
139
    ] = ev_data_df.loc[mask_work | mask_home, "charging_capacity_grid_MW"]
140
141
    ev_data_df["flex_last_timestep_charging_capacity_grid_MW"] = 0
142
    ev_data_df.loc[
143
        mask_work | mask_home, "flex_last_timestep_charging_capacity_grid_MW"
144
    ] = ev_data_df.loc[
145
        mask_work | mask_home, "last_timestep_charging_capacity_grid_MW"
146
    ]
147
148
    # Check length of timeseries
149
    if len(ev_data_df.loc[ev_data_df.last_timestep > 35040]) > 0:
150
        print("    Warning: Trip data exceeds 1 year and is cropped.")
151
        # Correct last TS
152
        ev_data_df.loc[
153
            ev_data_df.last_timestep > 35040, "last_timestep"
154
        ] = 35040
155
156
    if DATASET_CFG["model_timeseries"]["reduce_memory"]:
157
        return reduce_mem_usage(ev_data_df)
158
159
    return ev_data_df
160
161
162
def generate_load_time_series(
163
    ev_data_df: pd.DataFrame,
164
    run_config: pd.DataFrame,
165
    scenario_data: pd.DataFrame,
166
) -> pd.DataFrame:
167
    """Calculate the load time series from the given trip data. A dumb
168
    charging strategy is assumed where each EV starts charging immediately
169
    after plugging it in. Simultaneously the flexible charging capacity is
170
    calculated.
171
172
    Parameters
173
    ----------
174
    ev_data_df : pd.DataFrame
175
        Full trip data
176
    run_config : pd.DataFrame
177
        simBEV metadata: run config
178
    scenario_data : pd.Dataframe
179
        EV per grid district
180
181
    Returns
182
    -------
183
    pd.DataFrame
184
        time series of the load and the flex potential
185
    """
186
    # Get duplicates dict
187
    profile_counter = Counter(scenario_data.ev_id)
188
189
    # instantiate timeindex
190
    timeindex = pd.date_range(
191
        start=dt.datetime.fromisoformat(f"{run_config.start_date} 00:00:00"),
192
        end=dt.datetime.fromisoformat(f"{run_config.end_date} 23:45:00")
193
        + dt.timedelta(minutes=int(run_config.stepsize)),
194
        freq=f"{int(run_config.stepsize)}Min",
195
    )
196
197
    load_time_series_df = pd.DataFrame(
198
        data=0.0,
199
        index=timeindex,
200
        columns=["load_time_series", "flex_time_series"],
201
    )
202
203
    load_time_series_array = np.zeros(len(load_time_series_df))
204
    flex_time_series_array = load_time_series_array.copy()
205
    simultaneous_plugged_in_charging_capacity = load_time_series_array.copy()
206
    simultaneous_plugged_in_charging_capacity_flex = (
207
        load_time_series_array.copy()
208
    )
209
    soc_min_absolute = load_time_series_array.copy()
210
    soc_max_absolute = load_time_series_array.copy()
211
    driving_load_time_series_array = load_time_series_array.copy()
212
213
    columns = [
214
        "ev_id",
215
        "drive_start",
216
        "drive_end",
217
        "park_start",
218
        "park_end",
219
        "charge_end",
220
        "charging_capacity_grid_MW",
221
        "last_timestep",
222
        "last_timestep_charging_capacity_grid_MW",
223
        "flex_charging_capacity_grid_MW",
224
        "flex_last_timestep_charging_capacity_grid_MW",
225
        "soc_start",
226
        "soc_end",
227
        "bat_cap",
228
        "location",
229
        "consumption",
230
    ]
231
232
    # iterate over charging events
233
    for (
234
        _,
235
        ev_id,
236
        drive_start,
237
        drive_end,
238
        start,
239
        park_end,
240
        end,
241
        cap,
242
        last_ts,
243
        last_ts_cap,
244
        flex_cap,
245
        flex_last_ts_cap,
246
        soc_start,
247
        soc_end,
248
        bat_cap,
249
        location,
250
        consumption,
251
    ) in ev_data_df[columns].itertuples():
252
        ev_count = profile_counter[ev_id]
253
254
        load_time_series_array[start:end] += cap * ev_count
255
        load_time_series_array[last_ts] += last_ts_cap * ev_count
256
257
        flex_time_series_array[start:end] += flex_cap * ev_count
258
        flex_time_series_array[last_ts] += flex_last_ts_cap * ev_count
259
260
        simultaneous_plugged_in_charging_capacity[start : park_end + 1] += (
261
            cap * ev_count
262
        )
263
        simultaneous_plugged_in_charging_capacity_flex[
264
            start : park_end + 1
265
        ] += (flex_cap * ev_count)
266
267
        # ====================================================
268
        # min and max SoC constraints of aggregated EV battery
269
        # ====================================================
270
        # (I) Preserve SoC while driving
271
        if location == "driving":
272
            # Full band while driving
273
            # soc_min_absolute[drive_start:drive_end+1] +=
274
            # soc_end * bat_cap * ev_count
275
            #
276
            # soc_max_absolute[drive_start:drive_end+1] +=
277
            # soc_start * bat_cap * ev_count
278
279
            # Real band (decrease SoC while driving)
280
            soc_min_absolute[drive_start : drive_end + 1] += (
281
                np.linspace(soc_start, soc_end, drive_end - drive_start + 2)[
282
                    1:
283
                ]
284
                * bat_cap
285
                * ev_count
286
            )
287
            soc_max_absolute[drive_start : drive_end + 1] += (
288
                np.linspace(soc_start, soc_end, drive_end - drive_start + 2)[
289
                    1:
290
                ]
291
                * bat_cap
292
                * ev_count
293
            )
294
295
            # Equal distribution of driving load
296
            if soc_start > soc_end:  # reqd. for PHEV
297
                driving_load_time_series_array[
298
                    drive_start : drive_end + 1
299
                ] += (consumption * ev_count) / (drive_end - drive_start + 1)
300
301
        # (II) Fix SoC bounds while parking w/o charging
302
        elif soc_start == soc_end:
303
            soc_min_absolute[start : park_end + 1] += (
304
                soc_start * bat_cap * ev_count
305
            )
306
            soc_max_absolute[start : park_end + 1] += (
307
                soc_end * bat_cap * ev_count
308
            )
309
310
        # (III) Set SoC bounds at start and end of parking while charging
311
        # for flexible and non-flexible events
312
        elif soc_start < soc_end:
313
            if flex_cap > 0:
314
                # * "flex" (private charging only, band: SoC_min..SoC_max)
315
                soc_min_absolute[start : park_end + 1] += (
316
                    soc_start * bat_cap * ev_count
317
                )
318
                soc_max_absolute[start : park_end + 1] += (
319
                    soc_end * bat_cap * ev_count
320
                )
321
322
                # * "flex+" (private charging only, band: 0..1)
323
                #   (IF USED: add elif with flex scenario)
324
                # soc_min_absolute[start] += soc_start * bat_cap * ev_count
325
                # soc_max_absolute[start] += soc_start * bat_cap * ev_count
326
                # soc_min_absolute[park_end] += soc_end * bat_cap * ev_count
327
                # soc_max_absolute[park_end] += soc_end * bat_cap * ev_count
328
329
            # * Set SoC bounds for non-flexible charging (increase SoC while
330
            #   charging)
331
            # (SKIP THIS PART for "flex++" (private+public charging))
332
            elif flex_cap == 0:
333
                soc_min_absolute[start : park_end + 1] += (
334
                    np.linspace(soc_start, soc_end, park_end - start + 1)
335
                    * bat_cap
336
                    * ev_count
337
                )
338
                soc_max_absolute[start : park_end + 1] += (
339
                    np.linspace(soc_start, soc_end, park_end - start + 1)
340
                    * bat_cap
341
                    * ev_count
342
                )
343
344
    # Build timeseries
345
    load_time_series_df = load_time_series_df.assign(
346
        load_time_series=load_time_series_array,
347
        flex_time_series=flex_time_series_array,
348
        simultaneous_plugged_in_charging_capacity=(
349
            simultaneous_plugged_in_charging_capacity
350
        ),
351
        simultaneous_plugged_in_charging_capacity_flex=(
352
            simultaneous_plugged_in_charging_capacity_flex
353
        ),
354
        soc_min_absolute=(soc_min_absolute / 1e3),
355
        soc_max_absolute=(soc_max_absolute / 1e3),
356
        driving_load_time_series=driving_load_time_series_array / 1e3,
357
    )
358
359
    # validate load timeseries
360
    np.testing.assert_almost_equal(
361
        load_time_series_df.load_time_series.sum() / 4,
362
        (
363
            ev_data_df.ev_id.apply(lambda _: profile_counter[_])
364
            * ev_data_df.charging_demand
365
        ).sum()
366
        / 1000
367
        / float(run_config.eta_cp),
368
        decimal=-1,
369
    )
370
371
    if DATASET_CFG["model_timeseries"]["reduce_memory"]:
372
        return reduce_mem_usage(load_time_series_df)
373
    return load_time_series_df
374
375
376
def generate_static_params(
377
    ev_data_df: pd.DataFrame,
378
    load_time_series_df: pd.DataFrame,
379
    evs_grid_district_df: pd.DataFrame,
380
) -> dict:
381
    """Calculate static parameters from trip data.
382
383
    * cumulative initial SoC
384
    * cumulative battery capacity
385
    * simultaneous plugged in charging capacity
386
387
    Parameters
388
    ----------
389
    ev_data_df : pd.DataFrame
390
        Fill trip data
391
392
    Returns
393
    -------
394
    dict
395
        Static parameters
396
    """
397
    max_df = (
398
        ev_data_df[["ev_id", "bat_cap", "charging_capacity_grid_MW"]]
399
        .groupby("ev_id")
400
        .max()
401
    )
402
403
    # Get EV duplicates dict and weight battery capacity
404
    max_df["bat_cap"] = max_df.bat_cap.mul(
405
        pd.Series(Counter(evs_grid_district_df.ev_id))
406
    )
407
408
    static_params_dict = {
409
        "store_ev_battery.e_nom_MWh": float(max_df.bat_cap.sum() / 1e3),
410
        "link_bev_charger.p_nom_MW": float(
411
            load_time_series_df.simultaneous_plugged_in_charging_capacity.max()
412
        ),
413
    }
414
415
    return static_params_dict
416
417
418
def load_evs_trips(
419
    scenario_name: str,
420
    evs_ids: list,
421
    charging_events_only: bool = False,
422
    flex_only_at_charging_events: bool = True,
423
) -> pd.DataFrame:
424
    """Load trips for EVs
425
426
    Parameters
427
    ----------
428
    scenario_name : str
429
        Scenario name
430
    evs_ids : list of int
431
        IDs of EV to load the trips for
432
    charging_events_only : bool
433
        Load only events where charging takes place
434
    flex_only_at_charging_events : bool
435
        Flexibility only at charging events. If False, flexibility is provided
436
        by plugged-in EVs even if no charging takes place.
437
438
    Returns
439
    -------
440
    pd.DataFrame
441
        Trip data
442
    """
443
    # Select only charigung events
444
    if charging_events_only is True:
445
        charging_condition = EgonEvTrip.charging_demand > 0
446
    else:
447
        charging_condition = EgonEvTrip.charging_demand >= 0
448
449
    with db.session_scope() as session:
450
        query = (
451
            session.query(
452
                EgonEvTrip.egon_ev_pool_ev_id.label("ev_id"),
453
                EgonEvTrip.location,
454
                EgonEvTrip.use_case,
455
                EgonEvTrip.charging_capacity_nominal,
456
                EgonEvTrip.charging_capacity_grid,
457
                EgonEvTrip.charging_capacity_battery,
458
                EgonEvTrip.soc_start,
459
                EgonEvTrip.soc_end,
460
                EgonEvTrip.charging_demand,
461
                EgonEvTrip.park_start,
462
                EgonEvTrip.park_end,
463
                EgonEvTrip.drive_start,
464
                EgonEvTrip.drive_end,
465
                EgonEvTrip.consumption,
466
                EgonEvPool.type,
467
            )
468
            .join(
469
                EgonEvPool, EgonEvPool.ev_id == EgonEvTrip.egon_ev_pool_ev_id
470
            )
471
            .filter(EgonEvTrip.egon_ev_pool_ev_id.in_(evs_ids))
472
            .filter(EgonEvTrip.scenario == scenario_name)
473
            .filter(EgonEvPool.scenario == scenario_name)
474
            .filter(charging_condition)
475
            .order_by(
476
                EgonEvTrip.egon_ev_pool_ev_id, EgonEvTrip.simbev_event_id
477
            )
478
        )
479
480
    trip_data = pd.read_sql(
481
        query.statement, query.session.bind, index_col=None
482
    ).astype(
483
        {
484
            "ev_id": "int",
485
            "park_start": "int",
486
            "park_end": "int",
487
            "drive_start": "int",
488
            "drive_end": "int",
489
        }
490
    )
491
492
    if flex_only_at_charging_events is True:
493
        # ASSUMPTION: set charging cap 0 where there's no demand
494
        # (discard other plugged-in times)
495
        mask = trip_data.charging_demand == 0
496
        trip_data.loc[mask, "charging_capacity_nominal"] = 0
497
        trip_data.loc[mask, "charging_capacity_grid"] = 0
498
        trip_data.loc[mask, "charging_capacity_battery"] = 0
499
500
    return trip_data
501
502
503
def write_model_data_to_db(
504
    static_params_dict: dict,
505
    load_time_series_df: pd.DataFrame,
506
    bus_id: int,
507
    scenario_name: str,
508
    run_config: pd.DataFrame,
509
    bat_cap: pd.DataFrame,
510
) -> None:
511
    """Write all results for grid district to database
512
513
    Parameters
514
    ----------
515
    static_params_dict : dict
516
        Static model params
517
    load_time_series_df : pd.DataFrame
518
        Load time series for grid district
519
    bus_id : int
520
        ID of grid district
521
    scenario_name : str
522
        Scenario name
523
    run_config : pd.DataFrame
524
        simBEV metadata: run config
525
    bat_cap : pd.DataFrame
526
        Battery capacities per EV type
527
528
    Returns
529
    -------
530
    None
531
    """
532
533
    def calc_initial_ev_soc(bus_id: int, scenario_name: str) -> pd.DataFrame:
534
        """Calculate an average initial state of charge for EVs in MV grid
535
        district.
536
537
        This is done by weighting the initial SoCs at timestep=0 with EV count
538
        and battery capacity for each EV type.
539
        """
540
        with db.session_scope() as session:
541
            query_ev_soc = (
542
                session.query(
543
                    EgonEvPool.type,
544
                    func.count(EgonEvTrip.egon_ev_pool_ev_id).label(
545
                        "ev_count"
546
                    ),
547
                    func.avg(EgonEvTrip.soc_start).label("ev_soc_start"),
548
                )
549
                .select_from(EgonEvTrip)
550
                .join(
551
                    EgonEvPool,
552
                    EgonEvPool.ev_id == EgonEvTrip.egon_ev_pool_ev_id,
553
                )
554
                .join(
555
                    EgonEvMvGridDistrict,
556
                    EgonEvMvGridDistrict.egon_ev_pool_ev_id
557
                    == EgonEvTrip.egon_ev_pool_ev_id,
558
                )
559
                .filter(
560
                    EgonEvTrip.scenario == scenario_name,
561
                    EgonEvPool.scenario == scenario_name,
562
                    EgonEvMvGridDistrict.scenario == scenario_name,
563
                    EgonEvMvGridDistrict.bus_id == bus_id,
564
                    EgonEvTrip.simbev_event_id == 0,
565
                )
566
                .group_by(EgonEvPool.type)
567
            )
568
569
        initial_soc_per_ev_type = pd.read_sql(
570
            query_ev_soc.statement, query_ev_soc.session.bind, index_col="type"
571
        )
572
573
        initial_soc_per_ev_type[
574
            "battery_capacity_sum"
575
        ] = initial_soc_per_ev_type.ev_count.multiply(bat_cap)
576
        initial_soc_per_ev_type[
577
            "ev_soc_start_abs"
578
        ] = initial_soc_per_ev_type.battery_capacity_sum.multiply(
579
            initial_soc_per_ev_type.ev_soc_start
580
        )
581
582
        return (
583
            initial_soc_per_ev_type.ev_soc_start_abs.sum()
584
            / initial_soc_per_ev_type.battery_capacity_sum.sum()
585
        )
586
587
    def write_to_db(write_lowflex_model: bool) -> None:
588
        """Write model data to eTraGo tables"""
589
590
        @db.check_db_unique_violation
591
        def write_bus(scenario_name: str) -> None:
592
            # eMob MIT bus
593
            emob_bus_id = db.next_etrago_id("bus")
594
            with db.session_scope() as session:
595
                session.add(
596
                    EgonPfHvBus(
597
                        scn_name=scenario_name,
598
                        bus_id=emob_bus_id,
599
                        v_nom=1,
600
                        carrier="Li ion",
601
                        x=etrago_bus.x,
602
                        y=etrago_bus.y,
603
                        geom=etrago_bus.geom,
604
                    )
605
                )
606
            return emob_bus_id
607
608
        @db.check_db_unique_violation
609
        def write_link(scenario_name: str) -> None:
610
            # eMob MIT link [bus_el] -> [bus_ev]
611
            emob_link_id = db.next_etrago_id("link")
612
            with db.session_scope() as session:
613
                session.add(
614
                    EgonPfHvLink(
615
                        scn_name=scenario_name,
616
                        link_id=emob_link_id,
617
                        bus0=etrago_bus.bus_id,
618
                        bus1=emob_bus_id,
619
                        carrier="BEV charger",
620
                        efficiency=float(run_config.eta_cp),
621
                        p_nom=(
622
                            load_time_series_df.simultaneous_plugged_in_charging_capacity.max()
623
                        ),
624
                        p_nom_extendable=False,
625
                        p_nom_min=0,
626
                        p_nom_max=np.Inf,
627
                        p_min_pu=0,
628
                        p_max_pu=1,
629
                        # p_set_fixed=0,
630
                        capital_cost=0,
631
                        marginal_cost=0,
632
                        length=0,
633
                        terrain_factor=1,
634
                    )
635
                )
636
            with db.session_scope() as session:
637
                session.add(
638
                    EgonPfHvLinkTimeseries(
639
                        scn_name=scenario_name,
640
                        link_id=emob_link_id,
641
                        temp_id=1,
642
                        p_min_pu=None,
643
                        p_max_pu=(
644
                            hourly_load_time_series_df.ev_availability.to_list()
645
                        ),
646
                    )
647
                )
648
649
        @db.check_db_unique_violation
650
        def write_store(scenario_name: str) -> None:
651
            # eMob MIT store
652
            emob_store_id = db.next_etrago_id("store")
653
            with db.session_scope() as session:
654
                session.add(
655
                    EgonPfHvStore(
656
                        scn_name=scenario_name,
657
                        store_id=emob_store_id,
658
                        bus=emob_bus_id,
659
                        carrier="battery storage",
660
                        e_nom=static_params_dict["store_ev_battery.e_nom_MWh"],
661
                        e_nom_extendable=False,
662
                        e_nom_min=0,
663
                        e_nom_max=np.Inf,
664
                        e_min_pu=0,
665
                        e_max_pu=1,
666
                        e_initial=(
667
                            initial_soc_mean
668
                            * static_params_dict["store_ev_battery.e_nom_MWh"]
669
                        ),
670
                        e_cyclic=False,
671
                        sign=1,
672
                        standing_loss=0,
673
                    )
674
                )
675
            with db.session_scope() as session:
676
                session.add(
677
                    EgonPfHvStoreTimeseries(
678
                        scn_name=scenario_name,
679
                        store_id=emob_store_id,
680
                        temp_id=1,
681
                        e_min_pu=hourly_load_time_series_df.soc_min.to_list(),
682
                        e_max_pu=hourly_load_time_series_df.soc_max.to_list(),
683
                    )
684
                )
685
686
        @db.check_db_unique_violation
687
        def write_load(
688
            scenario_name: str, connection_bus_id: int, load_ts: list
689
        ) -> None:
690
            # eMob MIT load
691
            emob_load_id = db.next_etrago_id("load")
692
            with db.session_scope() as session:
693
                session.add(
694
                    EgonPfHvLoad(
695
                        scn_name=scenario_name,
696
                        load_id=emob_load_id,
697
                        bus=connection_bus_id,
698
                        carrier="land transport EV",
699
                        sign=-1,
700
                    )
701
                )
702
            with db.session_scope() as session:
703
                session.add(
704
                    EgonPfHvLoadTimeseries(
705
                        scn_name=scenario_name,
706
                        load_id=emob_load_id,
707
                        temp_id=1,
708
                        p_set=load_ts,
709
                    )
710
                )
711
712
        # Get eTraGo substation bus
713
        with db.session_scope() as session:
714
            query = session.query(
715
                EgonPfHvBus.scn_name,
716
                EgonPfHvBus.bus_id,
717
                EgonPfHvBus.x,
718
                EgonPfHvBus.y,
719
                EgonPfHvBus.geom,
720
            ).filter(
721
                EgonPfHvBus.scn_name == scenario_name,
722
                EgonPfHvBus.bus_id == bus_id,
723
                EgonPfHvBus.carrier == "AC",
724
            )
725
            etrago_bus = query.first()
726
            if etrago_bus is None:
727
                # TODO: raise exception here!
728
                print(
729
                    f"No AC bus found for scenario {scenario_name} "
730
                    f"with bus_id {bus_id} in table egon_etrago_bus!"
731
                )
732
733
        # Call DB writing functions for regular or lowflex scenario
734
        # * use corresponding scenario name as defined in datasets.yml
735
        # * no storage for lowflex scenario
736
        # * load timeseries:
737
        #   * regular (flex): use driving load
738
        #   * lowflex: use dumb charging load
739
        if write_lowflex_model is False:
740
            emob_bus_id = write_bus(scenario_name=scenario_name)
741
            write_link(scenario_name=scenario_name)
742
            write_store(scenario_name=scenario_name)
743
            write_load(
744
                scenario_name=scenario_name,
745
                connection_bus_id=emob_bus_id,
746
                load_ts=(
747
                    hourly_load_time_series_df.driving_load_time_series.to_list()
748
                ),
749
            )
750
        else:
751
            # Get lowflex scenario name
752
            lowflex_scenario_name = DATASET_CFG["scenario"]["lowflex"][
753
                "names"
754
            ][scenario_name]
755
            write_load(
756
                scenario_name=lowflex_scenario_name,
757
                connection_bus_id=etrago_bus.bus_id,
758
                load_ts=hourly_load_time_series_df.load_time_series.to_list(),
759
            )
760
761
    def write_to_file():
762
        """Write model data to file (for debugging purposes)"""
763
        results_dir = WORKING_DIR / Path("results", scenario_name, str(bus_id))
764
        results_dir.mkdir(exist_ok=True, parents=True)
765
766
        hourly_load_time_series_df[["load_time_series"]].to_csv(
767
            results_dir / "ev_load_time_series.csv"
768
        )
769
        hourly_load_time_series_df[["ev_availability"]].to_csv(
770
            results_dir / "ev_availability.csv"
771
        )
772
        hourly_load_time_series_df[["soc_min", "soc_max"]].to_csv(
773
            results_dir / "ev_dsm_profile.csv"
774
        )
775
776
        static_params_dict[
777
            "load_land_transport_ev.p_set_MW"
778
        ] = "ev_load_time_series.csv"
779
        static_params_dict["link_bev_charger.p_max_pu"] = "ev_availability.csv"
780
        static_params_dict["store_ev_battery.e_min_pu"] = "ev_dsm_profile.csv"
781
        static_params_dict["store_ev_battery.e_max_pu"] = "ev_dsm_profile.csv"
782
783
        file = results_dir / "ev_static_params.json"
784
785
        with open(file, "w") as f:
786
            json.dump(static_params_dict, f, indent=4)
0 ignored issues
show
Comprehensibility Best Practice introduced by
The variable static_params_dict does not seem to be defined.
Loading history...
787
788
    print("  Writing model timeseries...")
789
    load_time_series_df = load_time_series_df.assign(
790
        ev_availability=(
791
            load_time_series_df.simultaneous_plugged_in_charging_capacity
792
            / static_params_dict["link_bev_charger.p_nom_MW"]
793
        )
794
    )
795
796
    # Resample to 1h
797
    hourly_load_time_series_df = load_time_series_df.resample("1H").agg(
798
        {
799
            "load_time_series": np.mean,
800
            "flex_time_series": np.mean,
801
            "simultaneous_plugged_in_charging_capacity": np.mean,
802
            "simultaneous_plugged_in_charging_capacity_flex": np.mean,
803
            "soc_min_absolute": np.min,
804
            "soc_max_absolute": np.max,
805
            "ev_availability": np.mean,
806
            "driving_load_time_series": np.sum,
807
        }
808
    )
809
810
    # Create relative SoC timeseries
811
    hourly_load_time_series_df = hourly_load_time_series_df.assign(
812
        soc_min=hourly_load_time_series_df.soc_min_absolute.div(
813
            static_params_dict["store_ev_battery.e_nom_MWh"]
814
        ),
815
        soc_max=hourly_load_time_series_df.soc_max_absolute.div(
816
            static_params_dict["store_ev_battery.e_nom_MWh"]
817
        ),
818
    )
819
    hourly_load_time_series_df = hourly_load_time_series_df.assign(
820
        soc_delta_absolute=(
821
            hourly_load_time_series_df.soc_max_absolute
822
            - hourly_load_time_series_df.soc_min_absolute
823
        ),
824
        soc_delta=(
825
            hourly_load_time_series_df.soc_max
826
            - hourly_load_time_series_df.soc_min
827
        ),
828
    )
829
830
    # Crop hourly TS if needed
831
    hourly_load_time_series_df = hourly_load_time_series_df[:8760]
832
833
    # Create lowflex scenario?
834
    write_lowflex_model = DATASET_CFG["scenario"]["lowflex"][
835
        "create_lowflex_scenario"
836
    ]
837
838
    # Get initial average storage SoC
839
    initial_soc_mean = calc_initial_ev_soc(bus_id, scenario_name)
840
841
    # Write to database: regular and lowflex scenario
842
    write_to_db(write_lowflex_model=False)
843
    print("    Writing flex scenario...")
844
    if write_lowflex_model is True:
845
        print("    Writing lowflex scenario...")
846
        write_to_db(write_lowflex_model=True)
847
848
    # Export to working dir if requested
849
    if DATASET_CFG["model_timeseries"]["export_results_to_csv"]:
850
        write_to_file()
851
852
853
def delete_model_data_from_db():
854
    """Delete all eMob MIT data from eTraGo PF tables"""
855
    with db.session_scope() as session:
856
        # Buses
857
        session.query(EgonPfHvBus).filter(
858
            EgonPfHvBus.carrier == "Li ion"
859
        ).delete(synchronize_session=False)
860
861
        # Link TS
862
        subquery = (
863
            session.query(EgonPfHvLink.link_id)
864
            .filter(EgonPfHvLink.carrier == "BEV charger")
865
            .subquery()
866
        )
867
868
        session.query(EgonPfHvLinkTimeseries).filter(
869
            EgonPfHvLinkTimeseries.link_id.in_(subquery)
870
        ).delete(synchronize_session=False)
871
        # Links
872
        session.query(EgonPfHvLink).filter(
873
            EgonPfHvLink.carrier == "BEV charger"
874
        ).delete(synchronize_session=False)
875
876
        # Store TS
877
        subquery = (
878
            session.query(EgonPfHvStore.store_id)
879
            .filter(EgonPfHvStore.carrier == "battery storage")
880
            .subquery()
881
        )
882
883
        session.query(EgonPfHvStoreTimeseries).filter(
884
            EgonPfHvStoreTimeseries.store_id.in_(subquery)
885
        ).delete(synchronize_session=False)
886
        # Stores
887
        session.query(EgonPfHvStore).filter(
888
            EgonPfHvStore.carrier == "battery storage"
889
        ).delete(synchronize_session=False)
890
891
        # Load TS
892
        subquery = (
893
            session.query(EgonPfHvLoad.load_id)
894
            .filter(EgonPfHvLoad.carrier == "land transport EV")
895
            .subquery()
896
        )
897
898
        session.query(EgonPfHvLoadTimeseries).filter(
899
            EgonPfHvLoadTimeseries.load_id.in_(subquery)
900
        ).delete(synchronize_session=False)
901
        # Loads
902
        session.query(EgonPfHvLoad).filter(
903
            EgonPfHvLoad.carrier == "land transport EV"
904
        ).delete(synchronize_session=False)
905
906
907
def load_grid_district_ids() -> pd.Series:
908
    """Load bus IDs of all grid districts"""
909
    with db.session_scope() as session:
910
        query_mvgd = session.query(MvGridDistricts.bus_id)
911
    return pd.read_sql(
912
        query_mvgd.statement, query_mvgd.session.bind, index_col=None
913
    ).bus_id.sort_values()
914
915
916
def generate_model_data_grid_district(
917
    scenario_name: str,
918
    evs_grid_district: pd.DataFrame,
919
    bat_cap_dict: dict,
920
    run_config: pd.DataFrame,
921
) -> tuple:
922
    """Generates timeseries from simBEV trip data for MV grid district
923
924
    Parameters
925
    ----------
926
    scenario_name : str
927
        Scenario name
928
    evs_grid_district : pd.DataFrame
929
        EV data for grid district
930
    bat_cap_dict : dict
931
        Battery capacity per EV type
932
    run_config : pd.DataFrame
933
        simBEV metadata: run config
934
935
    Returns
936
    -------
937
    pd.DataFrame
938
        Model data for grid district
939
    """
940
941
    # Load trip data
942
    print("  Loading trips...")
943
    trip_data = load_evs_trips(
944
        scenario_name=scenario_name,
945
        evs_ids=evs_grid_district.ev_id.unique(),
946
        charging_events_only=False,
947
        flex_only_at_charging_events=True,
948
    )
949
950
    print("  Preprocessing data...")
951
    # Assign battery capacity to trip data
952
    trip_data["bat_cap"] = trip_data.type.apply(lambda _: bat_cap_dict[_])
953
    trip_data.drop(columns=["type"], inplace=True)
954
955
    # Preprocess trip data
956
    trip_data = data_preprocessing(evs_grid_district, trip_data)
957
958
    # Generate load timeseries
959
    print("  Generating load timeseries...")
960
    load_ts = generate_load_time_series(
961
        ev_data_df=trip_data,
962
        run_config=run_config,
963
        scenario_data=evs_grid_district,
964
    )
965
966
    # Generate static params
967
    static_params = generate_static_params(
968
        trip_data, load_ts, evs_grid_district
969
    )
970
971
    return static_params, load_ts
972
973
974
def generate_model_data_bunch(scenario_name: str, bunch: range) -> None:
975
    """Generates timeseries from simBEV trip data for a bunch of MV grid
976
    districts.
977
978
    Parameters
979
    ----------
980
    scenario_name : str
981
        Scenario name
982
    bunch : list
983
        Bunch of grid districts to generate data for, e.g. [1,2,..,100].
984
        Note: `bunch` is NOT a list of grid districts but is used for slicing
985
        the ordered list (by bus_id) of grid districts! This is used for
986
        parallelization. See
987
        :meth:`egon.data.datasets.emobility.motorized_individual_travel.MotorizedIndividualTravel.generate_model_data_tasks`
988
    """
989
    # Get list of grid districts / substations for this bunch
990
    mvgd_bus_ids = load_grid_district_ids().iloc[bunch]
991
992
    # Get scenario variation name
993
    scenario_var_name = DATASET_CFG["scenario"]["variation"][scenario_name]
994
995
    print(
996
        f"SCENARIO: {scenario_name}, "
997
        f"SCENARIO VARIATION: {scenario_var_name}, "
998
        f"BUNCH: {bunch[0]}-{bunch[-1]}"
999
    )
1000
1001
    # Load scenario params for scenario and scenario variation
1002
    # scenario_variation_parameters = get_sector_parameters(
1003
    #    "mobility", scenario=scenario_name
1004
    # )["motorized_individual_travel"][scenario_var_name]
1005
1006
    # Get substations
1007
    with db.session_scope() as session:
1008
        query = (
1009
            session.query(
1010
                EgonEvMvGridDistrict.bus_id,
1011
                EgonEvMvGridDistrict.egon_ev_pool_ev_id.label("ev_id"),
1012
            )
1013
            .filter(EgonEvMvGridDistrict.scenario == scenario_name)
1014
            .filter(
1015
                EgonEvMvGridDistrict.scenario_variation == scenario_var_name
1016
            )
1017
            .filter(EgonEvMvGridDistrict.bus_id.in_(mvgd_bus_ids))
1018
            .filter(EgonEvMvGridDistrict.egon_ev_pool_ev_id.isnot(None))
1019
        )
1020
    evs_grid_district = pd.read_sql(
1021
        query.statement, query.session.bind, index_col=None
1022
    ).astype({"ev_id": "int"})
1023
1024
    mvgd_bus_ids = evs_grid_district.bus_id.unique()
1025
    print(
1026
        f"{len(evs_grid_district)} EV loaded "
1027
        f"({len(evs_grid_district.ev_id.unique())} unique) in "
1028
        f"{len(mvgd_bus_ids)} grid districts."
1029
    )
1030
1031
    # Get run metadata
1032
    meta_tech_data = read_simbev_metadata_file(scenario_name, "tech_data")
1033
    meta_run_config = read_simbev_metadata_file(scenario_name, "config").loc[
1034
        "basic"
1035
    ]
1036
1037
    # Generate timeseries for each MVGD
1038
    print("GENERATE MODEL DATA...")
1039
    ctr = 0
1040
    for bus_id in mvgd_bus_ids:
1041
        ctr += 1
1042
        print(
1043
            f"Processing grid district: bus {bus_id}... "
1044
            f"({ctr}/{len(mvgd_bus_ids)})"
1045
        )
1046
        (static_params, load_ts,) = generate_model_data_grid_district(
1047
            scenario_name=scenario_name,
1048
            evs_grid_district=evs_grid_district[
1049
                evs_grid_district.bus_id == bus_id
1050
            ],
1051
            bat_cap_dict=meta_tech_data.battery_capacity.to_dict(),
1052
            run_config=meta_run_config,
1053
        )
1054
        write_model_data_to_db(
1055
            static_params_dict=static_params,
1056
            load_time_series_df=load_ts,
1057
            bus_id=bus_id,
1058
            scenario_name=scenario_name,
1059
            run_config=meta_run_config,
1060
            bat_cap=meta_tech_data.battery_capacity,
1061
        )
1062
1063
1064
def generate_model_data_eGon2035_remaining():
1065
    """Generates timeseries for eGon2035 scenario for grid districts which
1066
    has not been processed in the parallel tasks before.
1067
    """
1068
    generate_model_data_bunch(
1069
        scenario_name="eGon2035",
1070
        bunch=range(MVGD_MIN_COUNT, len(load_grid_district_ids())),
1071
    )
1072
1073
1074
def generate_model_data_eGon100RE_remaining():
1075
    """Generates timeseries for eGon100RE scenario for grid districts which
1076
    has not been processed in the parallel tasks before.
1077
    """
1078
    generate_model_data_bunch(
1079
        scenario_name="eGon100RE",
1080
        bunch=range(MVGD_MIN_COUNT, len(load_grid_district_ids())),
1081
    )
1082