data.datasets.sanity_checks.sanitycheck_eGon2035_electricity() - Code Metrics - Inspection of "Features/#684 distribute pv rooftop buildings 2" - openego/eGon-data - Measure and Improve Code Quality continuously with Scrutinizer

Passed

Pull Request — dev (#809)

unknown

created 2022-08-24 07:56 UTC

sanitycheck_eGon2035_electricity() F

↳ Parent: data.datasets.sanity_checks

Complexity

Conditions

Size

Total Lines	231
Code Lines	97

Duplication

Lines	64
Ratio	27.71 %

Importance

Changes

Metric	Value
eloc	97
dl	64
loc	231
rs	1.7781
c	0
b	0
f	0
cc	16
nop	0

How to fix Long Method Complexity

Long Method

Small methods make your code easier to understand, in particular if combined with a good name. Besides, if your method is small, finding a good name is usually much easier.

For example, if you find yourself adding comments to a method's body, this is usually a good sign to extract the commented part to a new method, and use the comment as a starting point when coming up with a good name for this new method.

Commonly applied refactorings include:

If many parameters/temporary variables are present:
- Replace temporary variables with Query
- Introduce parameter object; often combined with preserve whole object
- If the above two are insufficient: Replace method with method object
If you have long conditionals: Decompose Conditional
Otherwise: Extract method

Complexity

Complex classes like data.datasets.sanity_checks.sanitycheck_eGon2035_electricity() often do a lot of different things. To break such a class down, we need to identify a cohesive component within that class. A common approach to find such a component is to look for fields/methods that share the same prefixes, or suffixes.

Once you have determined the fields that belong together, you can apply the Extract Class refactoring. If the component makes sense as a sub-class, Extract Subclass is also a candidate, and is often faster.

"""
This module does sanity checks for both the eGon2035 and the eGon100RE scenario seperately where a percentage
error is given to showcase difference in output and input values. Please note that there are missing input technologies in the supply tables.
 Authors: @ALonso, @dana
"""
from math import isclose
from pathlib import Path

from loguru import logger
import matplotlib.pyplot as plt
import seaborn as sns

from egon.data import db
from egon.data.datasets import Dataset
from egon.data.datasets.power_plants.pv_rooftop_buildings import (
    municipality_data,
    osm_buildings,
    scenario_data,
)
import egon.data


class SanityChecks(Dataset):
    def __init__(self, dependencies):
        super().__init__(
            name="SanityChecks",
            version="0.0.2",
            dependencies=dependencies,
            tasks=(
                sanitycheck_eGon2035_electricity,
                sanitycheck_eGon2035_heat,
                sanitycheck_pv_rooftop_buildings,
            ),
        )


def sanitycheck_eGon2035_electricity():
    """Execute basic sanity checks.

    Returns print statements as sanity checks for the electricity sector in
    the eGon2035 scenario.

    Parameters
    ----------
    None

    Returns
    -------
    None
    """

    scn = "eGon2035"

    # Section to check generator capacities
    print(f"Sanity checks for scenario {scn}")
    print(
        "For German electricity generators the following deviations between the inputs and outputs can be observed:"
    )

    carriers_electricity = [
        "other_non_renewable",
        "other_renewable",
        "reservoir",
        "run_of_river",
        "oil",
        "wind_onshore",
        "wind_offshore",
        "solar",
        "solar_rooftop",
        "biomass",
    ]

    for carrier in carriers_electricity:

        if carrier == "biomass":
            sum_output = db.select_dataframe(
                """SELECT scn_name, SUM(p_nom::numeric) as output_capacity_mw
                    FROM grid.egon_etrago_generator
                    WHERE bus IN (
                        SELECT bus_id FROM grid.egon_etrago_bus
                        WHERE scn_name = 'eGon2035'
                        AND country = 'DE')
                    AND carrier IN ('biomass', 'industrial_biomass_CHP', 'central_biomass_CHP')
                    GROUP BY (scn_name);
                """,
                warning=False,
            )

        else:
            sum_output = db.select_dataframe(
                f"""SELECT scn_name, SUM(p_nom::numeric) as output_capacity_mw
                         FROM grid.egon_etrago_generator
                         WHERE scn_name = '{scn}'
                         AND carrier IN ('{carrier}')
                         AND bus IN
                             (SELECT bus_id
                               FROM grid.egon_etrago_bus
                               WHERE scn_name = 'eGon2035'
                               AND country = 'DE')
                         GROUP BY (scn_name);
                    """,
                warning=False,
            )

        sum_input = db.select_dataframe(
            f"""SELECT carrier, SUM(capacity::numeric) as input_capacity_mw
                     FROM supply.egon_scenario_capacities
                     WHERE carrier= '{carrier}'
                     AND scenario_name ='{scn}'
                     GROUP BY (carrier);
                """,
            warning=False,
        )

        if (

            sum_output.output_capacity_mw.sum() == 0
            and sum_input.input_capacity_mw.sum() == 0
        ):
            print(
                f"No capacity for carrier '{carrier}' needed to be distributed. "
                f"Everything is fine"
            )

        elif (
            sum_input.input_capacity_mw.sum() > 0
            and sum_output.output_capacity_mw.sum() == 0
        ):
            print(
                f"Error: Capacity for carrier '{carrier}' was not distributed at all!"
            )

        elif (
            sum_output.output_capacity_mw.sum() > 0
            and sum_input.input_capacity_mw.sum() == 0
        ):
            print(
                f"Error: Eventhough no input capacity was provided for carrier '{carrier}' a capacity got distributed!"
            )

        else:
            sum_input["error"] = (
                (sum_output.output_capacity_mw - sum_input.input_capacity_mw)
                / sum_input.input_capacity_mw
            ) * 100
            g = sum_input["error"].values[0]

            print(f"{carrier}: " + str(round(g, 2)) + " %")

    # Section to check storage units

    print(f"Sanity checks for scenario {scn}")
    print(
        "For German electrical storage units the following deviations between the inputs and outputs can be observed:"
    )

    carriers_electricity = ["pumped_hydro"]

    for carrier in carriers_electricity:

        sum_output = db.select_dataframe(
            f"""SELECT scn_name, SUM(p_nom::numeric) as output_capacity_mw
                         FROM grid.egon_etrago_storage
                         WHERE scn_name = '{scn}'
                         AND carrier IN ('{carrier}')
                         AND bus IN
                             (SELECT bus_id
                               FROM grid.egon_etrago_bus
                               WHERE scn_name = 'eGon2035'
                               AND country = 'DE')
                         GROUP BY (scn_name);
                    """,
            warning=False,
        )

        sum_input = db.select_dataframe(
            f"""SELECT carrier, SUM(capacity::numeric) as input_capacity_mw
                     FROM supply.egon_scenario_capacities
                     WHERE carrier= '{carrier}'
                     AND scenario_name ='{scn}'
                     GROUP BY (carrier);
                """,
            warning=False,
        )

        if (

            sum_output.output_capacity_mw.sum() == 0
            and sum_input.input_capacity_mw.sum() == 0
        ):
            print(
                f"No capacity for carrier '{carrier}' needed to be distributed. Everything is fine"
            )

        elif (
            sum_input.input_capacity_mw.sum() > 0
            and sum_output.output_capacity_mw.sum() == 0
        ):
            print(
                f"Error: Capacity for carrier '{carrier}' was not distributed at all!"
            )

        elif (
            sum_output.output_capacity_mw.sum() > 0
            and sum_input.input_capacity_mw.sum() == 0
        ):
            print(
                f"Error: Eventhough no input capacity was provided for carrier '{carrier}' a capacity got distributed!"
            )

        else:
            sum_input["error"] = (
                (sum_output.output_capacity_mw - sum_input.input_capacity_mw)
                / sum_input.input_capacity_mw
            ) * 100
            g = sum_input["error"].values[0]

            print(f"{carrier}: " + str(round(g, 2)) + " %")

    # Section to check loads

    print(
        "For German electricity loads the following deviations between the input and output can be observed:"
    )

    output_demand = db.select_dataframe(
        """SELECT a.scn_name, a.carrier,  SUM((SELECT SUM(p) FROM UNNEST(b.p_set) p))/1000000::numeric as load_twh
            FROM grid.egon_etrago_load a
            JOIN grid.egon_etrago_load_timeseries b
            ON (a.load_id = b.load_id)
            JOIN grid.egon_etrago_bus c
            ON (a.bus=c.bus_id)
            AND b.scn_name = 'eGon2035'
            AND a.scn_name = 'eGon2035'
            AND a.carrier = 'AC'
            AND c.scn_name= 'eGon2035'
            AND c.country='DE'
            GROUP BY (a.scn_name, a.carrier);

    """,
        warning=False,
    )["load_twh"].values[0]

    input_cts_ind = db.select_dataframe(
        """SELECT scenario, SUM(demand::numeric/1000000) as demand_mw_regio_cts_ind
            FROM demand.egon_demandregio_cts_ind
            WHERE scenario= 'eGon2035'
            AND year IN ('2035')
            GROUP BY (scenario);

        """,
        warning=False,
    )["demand_mw_regio_cts_ind"].values[0]

    input_hh = db.select_dataframe(
        """SELECT scenario, SUM(demand::numeric/1000000) as demand_mw_regio_hh
            FROM demand.egon_demandregio_hh
            WHERE scenario= 'eGon2035'
            AND year IN ('2035')
            GROUP BY (scenario);
        """,
        warning=False,
    )["demand_mw_regio_hh"].values[0]

    input_demand = input_hh + input_cts_ind

    e = round((output_demand - input_demand) / input_demand, 2) * 100

    print(f"electricity demand: {e} %")


def sanitycheck_eGon2035_heat():
    """Execute basic sanity checks.

    Returns print statements as sanity checks for the heat sector in
    the eGon2035 scenario.

    Parameters
    ----------
    None

    Returns
    -------
    None
    """

    # Check input and output values for the carriers "other_non_renewable",
    # "other_renewable", "reservoir", "run_of_river" and "oil"

    scn = "eGon2035"

    # Section to check generator capacities
    print(f"Sanity checks for scenario {scn}")
    print(
        "For German heat demands the following deviations between the inputs and outputs can be observed:"
    )

    # Sanity checks for heat demand

    output_heat_demand = db.select_dataframe(
        """SELECT a.scn_name,  (SUM((SELECT SUM(p) FROM UNNEST(b.p_set) p))/1000000)::numeric as load_twh
            FROM grid.egon_etrago_load a
            JOIN grid.egon_etrago_load_timeseries b
            ON (a.load_id = b.load_id)
            JOIN grid.egon_etrago_bus c
            ON (a.bus=c.bus_id)
            AND b.scn_name = 'eGon2035'
            AND a.scn_name = 'eGon2035'
            AND c.scn_name= 'eGon2035'
            AND c.country='DE'
            AND a.carrier IN ('rural_heat', 'central_heat')
            GROUP BY (a.scn_name);
        """,
        warning=False,
    )["load_twh"].values[0]

    input_heat_demand = db.select_dataframe(
        """SELECT scenario, SUM(demand::numeric/1000000) as demand_mw_peta_heat
            FROM demand.egon_peta_heat
            WHERE scenario= 'eGon2035'
            GROUP BY (scenario);
        """,
        warning=False,
    )["demand_mw_peta_heat"].values[0]

    e_demand = (
        round((output_heat_demand - input_heat_demand) / input_heat_demand, 2)
        * 100
    )

    print(f"heat demand: {e_demand} %")

    # Sanity checks for heat supply

    print(
        "For German heat supplies the following deviations between the inputs and "
        "outputs can be observed:"
    )

    # Comparison for central heat pumps
    heat_pump_input = db.select_dataframe(
        """SELECT carrier, SUM(capacity::numeric) as Urban_central_heat_pump_mw
            FROM supply.egon_scenario_capacities
            WHERE carrier= 'urban_central_heat_pump'
            AND scenario_name IN ('eGon2035')
            GROUP BY (carrier);
        """,
        warning=False,
    )["urban_central_heat_pump_mw"].values[0]

    heat_pump_output = db.select_dataframe(
        """SELECT carrier, SUM(p_nom::numeric) as Central_heat_pump_mw
            FROM grid.egon_etrago_link
            WHERE carrier= 'central_heat_pump'
            AND scn_name IN ('eGon2035')
            GROUP BY (carrier);
    """,
        warning=False,
    )["central_heat_pump_mw"].values[0]

    e_heat_pump = (
        round((heat_pump_output - heat_pump_input) / heat_pump_output, 2) * 100
    )

    print(f"'central_heat_pump': {e_heat_pump} % ")

    # Comparison for residential heat pumps

    input_residential_heat_pump = db.select_dataframe(
        """SELECT carrier, SUM(capacity::numeric) as residential_heat_pump_mw
            FROM supply.egon_scenario_capacities
            WHERE carrier= 'residential_rural_heat_pump'
            AND scenario_name IN ('eGon2035')
            GROUP BY (carrier);
        """,
        warning=False,
    )["residential_heat_pump_mw"].values[0]

    output_residential_heat_pump = db.select_dataframe(
        """SELECT carrier, SUM(p_nom::numeric) as rural_heat_pump_mw
            FROM grid.egon_etrago_link
            WHERE carrier= 'rural_heat_pump'
            AND scn_name IN ('eGon2035')
            GROUP BY (carrier);
    """,
        warning=False,
    )["rural_heat_pump_mw"].values[0]

    e_residential_heat_pump = (
        round(
            (output_residential_heat_pump - input_residential_heat_pump)
            / input_residential_heat_pump,
            2,
        )
        * 100
    )
    print(f"'residential heat pumps': {e_residential_heat_pump} %")

    # Comparison for resistive heater
    resistive_heater_input = db.select_dataframe(
        """SELECT carrier, SUM(capacity::numeric) as Urban_central_resistive_heater_MW
            FROM supply.egon_scenario_capacities
            WHERE carrier= 'urban_central_resistive_heater'
            AND scenario_name IN ('eGon2035')
            GROUP BY (carrier);
        """,
        warning=False,
    )["urban_central_resistive_heater_mw"].values[0]

    resistive_heater_output = db.select_dataframe(
        """SELECT carrier, SUM(p_nom::numeric) as central_resistive_heater_MW
            FROM grid.egon_etrago_link
            WHERE carrier= 'central_resistive_heater'
            AND scn_name IN ('eGon2035')
            GROUP BY (carrier);
        """,
        warning=False,
    )["central_resistive_heater_mw"].values[0]

    e_resistive_heater = (
        round(
            (resistive_heater_output - resistive_heater_input)
            / resistive_heater_input,
            2,
        )
        * 100
    )

    print(f"'resistive heater': {e_resistive_heater} %")

    # Comparison for solar thermal collectors

    input_solar_thermal = db.select_dataframe(
        """SELECT carrier, SUM(capacity::numeric) as solar_thermal_collector_mw
            FROM supply.egon_scenario_capacities
            WHERE carrier= 'urban_central_solar_thermal_collector'
            AND scenario_name IN ('eGon2035')
            GROUP BY (carrier);
        """,
        warning=False,
    )["solar_thermal_collector_mw"].values[0]

    output_solar_thermal = db.select_dataframe(
        """SELECT carrier, SUM(p_nom::numeric) as solar_thermal_collector_mw
            FROM grid.egon_etrago_generator
            WHERE carrier= 'solar_thermal_collector'
            AND scn_name IN ('eGon2035')
            GROUP BY (carrier);
        """,
        warning=False,
    )["solar_thermal_collector_mw"].values[0]

    e_solar_thermal = (
        round(
            (output_solar_thermal - input_solar_thermal) / input_solar_thermal,
            2,
        )
        * 100
    )
    print(f"'solar thermal collector': {e_solar_thermal} %")

    # Comparison for geothermal

    input_geo_thermal = db.select_dataframe(
        """SELECT carrier, SUM(capacity::numeric) as Urban_central_geo_thermal_MW
            FROM supply.egon_scenario_capacities
            WHERE carrier= 'urban_central_geo_thermal'
            AND scenario_name IN ('eGon2035')
            GROUP BY (carrier);
        """,
        warning=False,
    )["urban_central_geo_thermal_mw"].values[0]

    output_geo_thermal = db.select_dataframe(
        """SELECT carrier, SUM(p_nom::numeric) as geo_thermal_MW
            FROM grid.egon_etrago_generator
            WHERE carrier= 'geo_thermal'
            AND scn_name IN ('eGon2035')
            GROUP BY (carrier);
    """,
        warning=False,
    )["geo_thermal_mw"].values[0]

    e_geo_thermal = (
        round((output_geo_thermal - input_geo_thermal) / input_geo_thermal, 2)
        * 100
    )
    print(f"'geothermal': {e_geo_thermal} %")


def sanitycheck_pv_rooftop_buildings():
    def egon_power_plants_pv_roof_building():
        sql = """
        SELECT *
        FROM supply.egon_power_plants_pv_roof_building
        """

        return db.select_dataframe(sql, index_col="index")

    pv_roof_df = egon_power_plants_pv_roof_building()

    municipalities_gdf = municipality_data()

    osm_buildings_gdf = osm_buildings(municipalities_gdf.crs)

    merge_df = pv_roof_df.merge(
        osm_buildings_gdf[["area"]],
        how="left",
        left_on="building_id",
        right_index=True,
    )

    assert len(merge_df.loc[merge_df.area.isna()]) == 0

    scenarios = ["status_quo", "eGon2035"]

    base_path = Path(egon.data.__path__[0]).resolve()

    res_dir = base_path / "sanity_checks"

    res_dir.mkdir(parents=True, exist_ok=True)

    for scenario in scenarios:
        fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(15, 8))

        scenario_df = merge_df.loc[merge_df.scenario == scenario]

        logger.info(
            scenario + " Capacity:\n" + str(scenario_df.capacity.describe())
        )

        small_gens_df = scenario_df.loc[scenario_df.capacity < 100]

        sns.histplot(data=small_gens_df, x="capacity", ax=ax1).set_title(
            scenario
        )

        sns.scatterplot(
            data=small_gens_df, x="capacity", y="area", ax=ax2
        ).set_title(scenario)

        plt.tight_layout()

        plt.savefig(
            res_dir / f"{scenario}_pv_rooftop_distribution.png",
            bbox_inches="tight",
        )

    scenarios = ["eGon2035"]  # "eGon100RE"]

    for scenario in scenarios:
        assert isclose(
            scenario_data(scenario=scenario).capacity.sum() * 1000,
            merge_df.loc[merge_df.scenario == scenario].capacity.sum(),
            rel_tol=1e-02,
        ), (
            f"{scenario_data(scenario=scenario).capacity.sum() * 1000} != "
            f"{merge_df.loc[merge_df.scenario == scenario].capacity.sum()}"
        )


1		"""
2		This module does sanity checks for both the eGon2035 and the eGon100RE scenario seperately where a percentage
3		error is given to showcase difference in output and input values. Please note that there are missing input technologies in the supply tables.
4		Authors: @ALonso, @dana
5		"""
6		from math import isclose
7		from pathlib import Path
8
9		from loguru import logger
10		import matplotlib.pyplot as plt
11		import seaborn as sns
12
13		from egon.data import db
14		from egon.data.datasets import Dataset
15		from egon.data.datasets.power_plants.pv_rooftop_buildings import (
16		municipality_data,
17		osm_buildings,
18		scenario_data,
19		)
20		import egon.data
21
22
23		class SanityChecks(Dataset):
24		def __init__(self, dependencies):
25		super().__init__(
26		name="SanityChecks",
27		version="0.0.2",
28		dependencies=dependencies,
29		tasks=(
30		sanitycheck_eGon2035_electricity,
31		sanitycheck_eGon2035_heat,
32		sanitycheck_pv_rooftop_buildings,
33		),
34		)
35
36
37		def sanitycheck_eGon2035_electricity():
38		"""Execute basic sanity checks.
39
40		Returns print statements as sanity checks for the electricity sector in
41		the eGon2035 scenario.
42
43		Parameters
44		----------
45		None
46
47		Returns
48		-------
49		None
50		"""
51
52		scn = "eGon2035"
53
54		# Section to check generator capacities
55		print(f"Sanity checks for scenario {scn}")
56		print(
57		"For German electricity generators the following deviations between the inputs and outputs can be observed:"
58		)
59
60		carriers_electricity = [
61		"other_non_renewable",
62		"other_renewable",
63		"reservoir",
64		"run_of_river",
65		"oil",
66		"wind_onshore",
67		"wind_offshore",
68		"solar",
69		"solar_rooftop",
70		"biomass",
71		]
72
73		for carrier in carriers_electricity:
74
75		if carrier == "biomass":
76		sum_output = db.select_dataframe(
77		"""SELECT scn_name, SUM(p_nom::numeric) as output_capacity_mw
78		FROM grid.egon_etrago_generator
79		WHERE bus IN (
80		SELECT bus_id FROM grid.egon_etrago_bus
81		WHERE scn_name = 'eGon2035'
82		AND country = 'DE')
83		AND carrier IN ('biomass', 'industrial_biomass_CHP', 'central_biomass_CHP')
84		GROUP BY (scn_name);
85		""",
86		warning=False,
87		)
88
89		else:
90		sum_output = db.select_dataframe(
91		f"""SELECT scn_name, SUM(p_nom::numeric) as output_capacity_mw
92		FROM grid.egon_etrago_generator
93		WHERE scn_name = '{scn}'
94		AND carrier IN ('{carrier}')
95		AND bus IN
96		(SELECT bus_id
97		FROM grid.egon_etrago_bus
98		WHERE scn_name = 'eGon2035'
99		AND country = 'DE')
100		GROUP BY (scn_name);
101		""",
102		warning=False,
103		)
104
105		sum_input = db.select_dataframe(
106		f"""SELECT carrier, SUM(capacity::numeric) as input_capacity_mw
107		FROM supply.egon_scenario_capacities
108		WHERE carrier= '{carrier}'
109		AND scenario_name ='{scn}'
110		GROUP BY (carrier);
111		""",
112		warning=False,
113		)
114
115	View Code Duplication	if (
		0 ignored issues – show Duplication introduced 2022-06-03 07:55 UTC by Report Bug Copy Issue Report This code seems to be duplicated in your project. Loading history...
116		sum_output.output_capacity_mw.sum() == 0
117		and sum_input.input_capacity_mw.sum() == 0
118		):
119		print(
120		f"No capacity for carrier '{carrier}' needed to be distributed. "
121		f"Everything is fine"
122		)
123
124		elif (
125		sum_input.input_capacity_mw.sum() > 0
126		and sum_output.output_capacity_mw.sum() == 0
127		):
128		print(
129		f"Error: Capacity for carrier '{carrier}' was not distributed at all!"
130		)
131
132		elif (
133		sum_output.output_capacity_mw.sum() > 0
134		and sum_input.input_capacity_mw.sum() == 0
135		):
136		print(
137		f"Error: Eventhough no input capacity was provided for carrier '{carrier}' a capacity got distributed!"
138		)
139
140		else:
141		sum_input["error"] = (
142		(sum_output.output_capacity_mw - sum_input.input_capacity_mw)
143		/ sum_input.input_capacity_mw
144		) * 100
145		g = sum_input["error"].values[0]
146
147		print(f"{carrier}: " + str(round(g, 2)) + " %")
148
149		# Section to check storage units
150
151		print(f"Sanity checks for scenario {scn}")
152		print(
153		"For German electrical storage units the following deviations between the inputs and outputs can be observed:"
154		)
155
156		carriers_electricity = ["pumped_hydro"]
157
158		for carrier in carriers_electricity:
159
160		sum_output = db.select_dataframe(
161		f"""SELECT scn_name, SUM(p_nom::numeric) as output_capacity_mw
162		FROM grid.egon_etrago_storage
163		WHERE scn_name = '{scn}'
164		AND carrier IN ('{carrier}')
165		AND bus IN
166		(SELECT bus_id
167		FROM grid.egon_etrago_bus
168		WHERE scn_name = 'eGon2035'
169		AND country = 'DE')
170		GROUP BY (scn_name);
171		""",
172		warning=False,
173		)
174
175		sum_input = db.select_dataframe(
176		f"""SELECT carrier, SUM(capacity::numeric) as input_capacity_mw
177		FROM supply.egon_scenario_capacities
178		WHERE carrier= '{carrier}'
179		AND scenario_name ='{scn}'
180		GROUP BY (carrier);
181		""",
182		warning=False,
183		)
184
185	View Code Duplication	if (
		0 ignored issues – show Duplication introduced 2022-06-03 07:55 UTC by Report Bug Copy Issue Report This code seems to be duplicated in your project. Loading history...
186		sum_output.output_capacity_mw.sum() == 0
187		and sum_input.input_capacity_mw.sum() == 0
188		):
189		print(
190		f"No capacity for carrier '{carrier}' needed to be distributed. Everything is fine"
191		)
192
193		elif (
194		sum_input.input_capacity_mw.sum() > 0
195		and sum_output.output_capacity_mw.sum() == 0
196		):
197		print(
198		f"Error: Capacity for carrier '{carrier}' was not distributed at all!"
199		)
200
201		elif (
202		sum_output.output_capacity_mw.sum() > 0
203		and sum_input.input_capacity_mw.sum() == 0
204		):
205		print(
206		f"Error: Eventhough no input capacity was provided for carrier '{carrier}' a capacity got distributed!"
207		)
208
209		else:
210		sum_input["error"] = (
211		(sum_output.output_capacity_mw - sum_input.input_capacity_mw)
212		/ sum_input.input_capacity_mw
213		) * 100
214		g = sum_input["error"].values[0]
215
216		print(f"{carrier}: " + str(round(g, 2)) + " %")
217
218		# Section to check loads
219
220		print(
221		"For German electricity loads the following deviations between the input and output can be observed:"
222		)
223
224		output_demand = db.select_dataframe(
225		"""SELECT a.scn_name, a.carrier, SUM((SELECT SUM(p) FROM UNNEST(b.p_set) p))/1000000::numeric as load_twh
226		FROM grid.egon_etrago_load a
227		JOIN grid.egon_etrago_load_timeseries b
228		ON (a.load_id = b.load_id)
229		JOIN grid.egon_etrago_bus c
230		ON (a.bus=c.bus_id)
231		AND b.scn_name = 'eGon2035'
232		AND a.scn_name = 'eGon2035'
233		AND a.carrier = 'AC'
234		AND c.scn_name= 'eGon2035'
235		AND c.country='DE'
236		GROUP BY (a.scn_name, a.carrier);
237
238		""",
239		warning=False,
240		)["load_twh"].values[0]
241
242		input_cts_ind = db.select_dataframe(
243		"""SELECT scenario, SUM(demand::numeric/1000000) as demand_mw_regio_cts_ind
244		FROM demand.egon_demandregio_cts_ind
245		WHERE scenario= 'eGon2035'
246		AND year IN ('2035')
247		GROUP BY (scenario);
248
249		""",
250		warning=False,
251		)["demand_mw_regio_cts_ind"].values[0]
252
253		input_hh = db.select_dataframe(
254		"""SELECT scenario, SUM(demand::numeric/1000000) as demand_mw_regio_hh
255		FROM demand.egon_demandregio_hh
256		WHERE scenario= 'eGon2035'
257		AND year IN ('2035')
258		GROUP BY (scenario);
259		""",
260		warning=False,
261		)["demand_mw_regio_hh"].values[0]
262
263		input_demand = input_hh + input_cts_ind
264
265		e = round((output_demand - input_demand) / input_demand, 2) * 100
266
267		print(f"electricity demand: {e} %")
268
269
270		def sanitycheck_eGon2035_heat():
271		"""Execute basic sanity checks.
272
273		Returns print statements as sanity checks for the heat sector in
274		the eGon2035 scenario.
275
276		Parameters
277		----------
278		None
279
280		Returns
281		-------
282		None
283		"""
284
285		# Check input and output values for the carriers "other_non_renewable",
286		# "other_renewable", "reservoir", "run_of_river" and "oil"
287
288		scn = "eGon2035"
289
290		# Section to check generator capacities
291		print(f"Sanity checks for scenario {scn}")
292		print(
293		"For German heat demands the following deviations between the inputs and outputs can be observed:"
294		)
295
296		# Sanity checks for heat demand
297
298		output_heat_demand = db.select_dataframe(
299		"""SELECT a.scn_name, (SUM((SELECT SUM(p) FROM UNNEST(b.p_set) p))/1000000)::numeric as load_twh
300		FROM grid.egon_etrago_load a
301		JOIN grid.egon_etrago_load_timeseries b
302		ON (a.load_id = b.load_id)
303		JOIN grid.egon_etrago_bus c
304		ON (a.bus=c.bus_id)
305		AND b.scn_name = 'eGon2035'
306		AND a.scn_name = 'eGon2035'
307		AND c.scn_name= 'eGon2035'
308		AND c.country='DE'
309		AND a.carrier IN ('rural_heat', 'central_heat')
310		GROUP BY (a.scn_name);
311		""",
312		warning=False,
313		)["load_twh"].values[0]
314
315		input_heat_demand = db.select_dataframe(
316		"""SELECT scenario, SUM(demand::numeric/1000000) as demand_mw_peta_heat
317		FROM demand.egon_peta_heat
318		WHERE scenario= 'eGon2035'
319		GROUP BY (scenario);
320		""",
321		warning=False,
322		)["demand_mw_peta_heat"].values[0]
323
324		e_demand = (
325		round((output_heat_demand - input_heat_demand) / input_heat_demand, 2)
326		* 100
327		)
328
329		print(f"heat demand: {e_demand} %")
330
331		# Sanity checks for heat supply
332
333		print(
334		"For German heat supplies the following deviations between the inputs and "
335		"outputs can be observed:"
336		)
337
338		# Comparison for central heat pumps
339		heat_pump_input = db.select_dataframe(
340		"""SELECT carrier, SUM(capacity::numeric) as Urban_central_heat_pump_mw
341		FROM supply.egon_scenario_capacities
342		WHERE carrier= 'urban_central_heat_pump'
343		AND scenario_name IN ('eGon2035')
344		GROUP BY (carrier);
345		""",
346		warning=False,
347		)["urban_central_heat_pump_mw"].values[0]
348
349		heat_pump_output = db.select_dataframe(
350		"""SELECT carrier, SUM(p_nom::numeric) as Central_heat_pump_mw
351		FROM grid.egon_etrago_link
352		WHERE carrier= 'central_heat_pump'
353		AND scn_name IN ('eGon2035')
354		GROUP BY (carrier);
355		""",
356		warning=False,
357		)["central_heat_pump_mw"].values[0]
358
359		e_heat_pump = (
360		round((heat_pump_output - heat_pump_input) / heat_pump_output, 2) * 100
361		)
362
363		print(f"'central_heat_pump': {e_heat_pump} % ")
364
365		# Comparison for residential heat pumps
366
367		input_residential_heat_pump = db.select_dataframe(
368		"""SELECT carrier, SUM(capacity::numeric) as residential_heat_pump_mw
369		FROM supply.egon_scenario_capacities
370		WHERE carrier= 'residential_rural_heat_pump'
371		AND scenario_name IN ('eGon2035')
372		GROUP BY (carrier);
373		""",
374		warning=False,
375		)["residential_heat_pump_mw"].values[0]
376
377		output_residential_heat_pump = db.select_dataframe(
378		"""SELECT carrier, SUM(p_nom::numeric) as rural_heat_pump_mw
379		FROM grid.egon_etrago_link
380		WHERE carrier= 'rural_heat_pump'
381		AND scn_name IN ('eGon2035')
382		GROUP BY (carrier);
383		""",
384		warning=False,
385		)["rural_heat_pump_mw"].values[0]
386
387		e_residential_heat_pump = (
388		round(
389		(output_residential_heat_pump - input_residential_heat_pump)
390		/ input_residential_heat_pump,
391		2,
392		)
393		* 100
394		)
395		print(f"'residential heat pumps': {e_residential_heat_pump} %")
396
397		# Comparison for resistive heater
398		resistive_heater_input = db.select_dataframe(
399		"""SELECT carrier, SUM(capacity::numeric) as Urban_central_resistive_heater_MW
400		FROM supply.egon_scenario_capacities
401		WHERE carrier= 'urban_central_resistive_heater'
402		AND scenario_name IN ('eGon2035')
403		GROUP BY (carrier);
404		""",
405		warning=False,
406		)["urban_central_resistive_heater_mw"].values[0]
407
408		resistive_heater_output = db.select_dataframe(
409		"""SELECT carrier, SUM(p_nom::numeric) as central_resistive_heater_MW
410		FROM grid.egon_etrago_link
411		WHERE carrier= 'central_resistive_heater'
412		AND scn_name IN ('eGon2035')
413		GROUP BY (carrier);
414		""",
415		warning=False,
416		)["central_resistive_heater_mw"].values[0]
417
418		e_resistive_heater = (
419		round(
420		(resistive_heater_output - resistive_heater_input)
421		/ resistive_heater_input,
422		2,
423		)
424		* 100
425		)
426
427		print(f"'resistive heater': {e_resistive_heater} %")
428
429		# Comparison for solar thermal collectors
430
431		input_solar_thermal = db.select_dataframe(
432		"""SELECT carrier, SUM(capacity::numeric) as solar_thermal_collector_mw
433		FROM supply.egon_scenario_capacities
434		WHERE carrier= 'urban_central_solar_thermal_collector'
435		AND scenario_name IN ('eGon2035')
436		GROUP BY (carrier);
437		""",
438		warning=False,
439		)["solar_thermal_collector_mw"].values[0]
440
441		output_solar_thermal = db.select_dataframe(
442		"""SELECT carrier, SUM(p_nom::numeric) as solar_thermal_collector_mw
443		FROM grid.egon_etrago_generator
444		WHERE carrier= 'solar_thermal_collector'
445		AND scn_name IN ('eGon2035')
446		GROUP BY (carrier);
447		""",
448		warning=False,
449		)["solar_thermal_collector_mw"].values[0]
450
451		e_solar_thermal = (
452		round(
453		(output_solar_thermal - input_solar_thermal) / input_solar_thermal,
454		2,
455		)
456		* 100
457		)
458		print(f"'solar thermal collector': {e_solar_thermal} %")
459
460		# Comparison for geothermal
461
462		input_geo_thermal = db.select_dataframe(
463		"""SELECT carrier, SUM(capacity::numeric) as Urban_central_geo_thermal_MW
464		FROM supply.egon_scenario_capacities
465		WHERE carrier= 'urban_central_geo_thermal'
466		AND scenario_name IN ('eGon2035')
467		GROUP BY (carrier);
468		""",
469		warning=False,
470		)["urban_central_geo_thermal_mw"].values[0]
471
472		output_geo_thermal = db.select_dataframe(
473		"""SELECT carrier, SUM(p_nom::numeric) as geo_thermal_MW
474		FROM grid.egon_etrago_generator
475		WHERE carrier= 'geo_thermal'
476		AND scn_name IN ('eGon2035')
477		GROUP BY (carrier);
478		""",
479		warning=False,
480		)["geo_thermal_mw"].values[0]
481
482		e_geo_thermal = (
483		round((output_geo_thermal - input_geo_thermal) / input_geo_thermal, 2)
484		* 100
485		)
486		print(f"'geothermal': {e_geo_thermal} %")
487
488
489		def sanitycheck_pv_rooftop_buildings():
490		def egon_power_plants_pv_roof_building():
491		sql = """
492		SELECT *
493		FROM supply.egon_power_plants_pv_roof_building
494		"""
495
496		return db.select_dataframe(sql, index_col="index")
497
498		pv_roof_df = egon_power_plants_pv_roof_building()
499
500		municipalities_gdf = municipality_data()
501
502		osm_buildings_gdf = osm_buildings(municipalities_gdf.crs)
503
504		merge_df = pv_roof_df.merge(
505		osm_buildings_gdf[["area"]],
506		how="left",
507		left_on="building_id",
508		right_index=True,
509		)
510
511		assert len(merge_df.loc[merge_df.area.isna()]) == 0
512
513		scenarios = ["status_quo", "eGon2035"]
514
515		base_path = Path(egon.data.__path__[0]).resolve()
516
517		res_dir = base_path / "sanity_checks"
518
519		res_dir.mkdir(parents=True, exist_ok=True)
520
521		for scenario in scenarios:
522		fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(15, 8))
523
524		scenario_df = merge_df.loc[merge_df.scenario == scenario]
525
526		logger.info(
527		scenario + " Capacity:\n" + str(scenario_df.capacity.describe())
528		)
529
530		small_gens_df = scenario_df.loc[scenario_df.capacity < 100]
531
532		sns.histplot(data=small_gens_df, x="capacity", ax=ax1).set_title(
533		scenario
534		)
535
536		sns.scatterplot(
537		data=small_gens_df, x="capacity", y="area", ax=ax2
538		).set_title(scenario)
539
540		plt.tight_layout()
541
542		plt.savefig(
543		res_dir / f"{scenario}_pv_rooftop_distribution.png",
544		bbox_inches="tight",
545		)
546
547		scenarios = ["eGon2035"] # "eGon100RE"]
548
549		for scenario in scenarios:
550		assert isclose(
551		scenario_data(scenario=scenario).capacity.sum() * 1000,
552		merge_df.loc[merge_df.scenario == scenario].capacity.sum(),
553		rel_tol=1e-02,
554		), (
555		f"{scenario_data(scenario=scenario).capacity.sum() * 1000} != "
556		f"{merge_df.loc[merge_df.scenario == scenario].capacity.sum()}"
557		)
558

openego / eGon-data

Pull Request — dev (#809)

sanitycheck_eGon2035_electricity() F

Complexity

Size

Duplication

Importance

How to fix Long Method Complexity

Long Method

Complexity

Duplication Side-by-Side

Filter issues like