data.datasets.sanity_checks.sanitycheck_eGon2035_electricity() - Code Metrics - Inspection of "Sanity checks for residential electricity loads" - openego/eGon-data - Measure and Improve Code Quality continuously with Scrutinizer

Passed

Pull Request — dev (#915)

unknown

created 2022-09-06 10:28 UTC

sanitycheck_eGon2035_electricity() F

↳ Parent: data.datasets.sanity_checks

Complexity

Conditions

Size

Total Lines	230
Code Lines	97

Duplication

Lines	64
Ratio	27.83 %

Importance

Changes

Metric	Value
eloc	97
dl	64
loc	230
rs	1.7781
c	0
b	0
f	0
cc	16
nop	0

How to fix Long Method Complexity

Long Method

Small methods make your code easier to understand, in particular if combined with a good name. Besides, if your method is small, finding a good name is usually much easier.

For example, if you find yourself adding comments to a method's body, this is usually a good sign to extract the commented part to a new method, and use the comment as a starting point when coming up with a good name for this new method.

Commonly applied refactorings include:

If many parameters/temporary variables are present:
- Replace temporary variables with Query
- Introduce parameter object; often combined with preserve whole object
- If the above two are insufficient: Replace method with method object
If you have long conditionals: Decompose Conditional
Otherwise: Extract method

Complexity

Complex classes like data.datasets.sanity_checks.sanitycheck_eGon2035_electricity() often do a lot of different things. To break such a class down, we need to identify a cohesive component within that class. A common approach to find such a component is to look for fields/methods that share the same prefixes, or suffixes.

Once you have determined the fields that belong together, you can apply the Extract Class refactoring. If the component makes sense as a sub-class, Extract Subclass is also a candidate, and is often faster.

"""
This module does sanity checks for both the eGon2035 and the eGon100RE scenario
separately where a percentage error is given to showcase difference in output
and input values. Please note that there are missing input technologies in the
supply tables.
Authors: @ALonso, @dana
"""

import numpy as np

from egon.data import db
from egon.data.datasets import Dataset


class SanityChecks(Dataset):
    def __init__(self, dependencies):
        super().__init__(
            name="SanityChecks",
            version="0.0.3",
            dependencies=dependencies,
            tasks={
                etrago_eGon2035_electricity,
                etrago_eGon2035_heat,
                residential_electricity_annual_sum,
                residential_electricity_hh_refinement,
            },
        )


def etrago_eGon2035_electricity():
    """Execute basic sanity checks.

    Returns print statements as sanity checks for the electricity sector in
    the eGon2035 scenario.

    Parameters
    ----------
    None

    Returns
    -------
    None
    """

    scn = "eGon2035"

    # Section to check generator capacities
    print(f"Sanity checks for scenario {scn}")
    print(
        "For German electricity generators the following deviations between "
        "the inputs and outputs can be observed:"
    )

    carriers_electricity = [
        "other_non_renewable",
        "other_renewable",
        "reservoir",
        "run_of_river",
        "oil",
        "wind_onshore",
        "wind_offshore",
        "solar",
        "solar_rooftop",
        "biomass",
    ]

    for carrier in carriers_electricity:

        if carrier == "biomass":
            sum_output = db.select_dataframe(
                """SELECT scn_name, SUM(p_nom::numeric) as output_capacity_mw
                    FROM grid.egon_etrago_generator
                    WHERE bus IN (
                        SELECT bus_id FROM grid.egon_etrago_bus
                        WHERE scn_name = 'eGon2035'
                        AND country = 'DE')
                    AND carrier IN ('biomass', 'industrial_biomass_CHP',
                    'central_biomass_CHP')
                    GROUP BY (scn_name);
                """,
                warning=False,
            )

        else:
            sum_output = db.select_dataframe(
                f"""SELECT scn_name,
                 SUM(p_nom::numeric) as output_capacity_mw
                         FROM grid.egon_etrago_generator
                         WHERE scn_name = '{scn}'
                         AND carrier IN ('{carrier}')
                         AND bus IN
                             (SELECT bus_id
                               FROM grid.egon_etrago_bus
                               WHERE scn_name = 'eGon2035'
                               AND country = 'DE')
                         GROUP BY (scn_name);
                    """,
                warning=False,
            )

        sum_input = db.select_dataframe(
            f"""SELECT carrier, SUM(capacity::numeric) as input_capacity_mw
                     FROM supply.egon_scenario_capacities
                     WHERE carrier= '{carrier}'
                     AND scenario_name ='{scn}'
                     GROUP BY (carrier);
                """,
            warning=False,
        )

        if (

            sum_output.output_capacity_mw.sum() == 0
            and sum_input.input_capacity_mw.sum() == 0
        ):
            print(
                f"No capacity for carrier '{carrier}' needed to be"
                f" distributed. Everything is fine"
            )

        elif (
            sum_input.input_capacity_mw.sum() > 0
            and sum_output.output_capacity_mw.sum() == 0
        ):
            print(
                f"Error: Capacity for carrier '{carrier}' was not distributed "
                f"at all!"
            )

        elif (
            sum_output.output_capacity_mw.sum() > 0
            and sum_input.input_capacity_mw.sum() == 0
        ):
            print(
                f"Error: Eventhough no input capacity was provided for carrier"
                f"'{carrier}' a capacity got distributed!"
            )

        else:
            sum_input["error"] = (
                (sum_output.output_capacity_mw - sum_input.input_capacity_mw)
                / sum_input.input_capacity_mw
            ) * 100
            g = sum_input["error"].values[0]

            print(f"{carrier}: " + str(round(g, 2)) + " %")

    # Section to check storage units

    print(f"Sanity checks for scenario {scn}")
    print(
        "For German electrical storage units the following deviations between"
        "the inputs and outputs can be observed:"
    )

    carriers_electricity = ["pumped_hydro"]

    for carrier in carriers_electricity:

        sum_output = db.select_dataframe(
            f"""SELECT scn_name, SUM(p_nom::numeric) as output_capacity_mw
                         FROM grid.egon_etrago_storage
                         WHERE scn_name = '{scn}'
                         AND carrier IN ('{carrier}')
                         AND bus IN
                             (SELECT bus_id
                               FROM grid.egon_etrago_bus
                               WHERE scn_name = 'eGon2035'
                               AND country = 'DE')
                         GROUP BY (scn_name);
                    """,
            warning=False,
        )

        sum_input = db.select_dataframe(
            f"""SELECT carrier, SUM(capacity::numeric) as input_capacity_mw
                     FROM supply.egon_scenario_capacities
                     WHERE carrier= '{carrier}'
                     AND scenario_name ='{scn}'
                     GROUP BY (carrier);
                """,
            warning=False,
        )

        if (

            sum_output.output_capacity_mw.sum() == 0
            and sum_input.input_capacity_mw.sum() == 0
        ):
            print(
                f"No capacity for carrier '{carrier}' needed to be "
                f"distributed. Everything is fine"
            )

        elif (
            sum_input.input_capacity_mw.sum() > 0
            and sum_output.output_capacity_mw.sum() == 0
        ):
            print(
                f"Error: Capacity for carrier '{carrier}' was not distributed"
                f" at all!"
            )

        elif (
            sum_output.output_capacity_mw.sum() > 0
            and sum_input.input_capacity_mw.sum() == 0
        ):
            print(
                f"Error: Eventhough no input capacity was provided for carrier"
                f" '{carrier}' a capacity got distributed!"
            )

        else:
            sum_input["error"] = (
                (sum_output.output_capacity_mw - sum_input.input_capacity_mw)
                / sum_input.input_capacity_mw
            ) * 100
            g = sum_input["error"].values[0]

            print(f"{carrier}: " + str(round(g, 2)) + " %")

    # Section to check loads

    print(
        "For German electricity loads the following deviations between the"
        " input and output can be observed:"
    )

    output_demand = db.select_dataframe(
        """SELECT a.scn_name, a.carrier,  SUM((SELECT SUM(p)
        FROM UNNEST(b.p_set) p))/1000000::numeric as load_twh
            FROM grid.egon_etrago_load a
            JOIN grid.egon_etrago_load_timeseries b
            ON (a.load_id = b.load_id)
            JOIN grid.egon_etrago_bus c
            ON (a.bus=c.bus_id)
            AND b.scn_name = 'eGon2035'
            AND a.scn_name = 'eGon2035'
            AND a.carrier = 'AC'
            AND c.scn_name= 'eGon2035'
            AND c.country='DE'
            GROUP BY (a.scn_name, a.carrier);

    """,
        warning=False,
    )["load_twh"].values[0]

    input_cts_ind = db.select_dataframe(
        """SELECT scenario,
         SUM(demand::numeric/1000000) as demand_mw_regio_cts_ind
            FROM demand.egon_demandregio_cts_ind
            WHERE scenario= 'eGon2035'
            AND year IN ('2035')
            GROUP BY (scenario);

        """,
        warning=False,
    )["demand_mw_regio_cts_ind"].values[0]

    input_hh = db.select_dataframe(
        """SELECT scenario, SUM(demand::numeric/1000000) as demand_mw_regio_hh
            FROM demand.egon_demandregio_hh
            WHERE scenario= 'eGon2035'
            AND year IN ('2035')
            GROUP BY (scenario);
        """,
        warning=False,
    )["demand_mw_regio_hh"].values[0]

    input_demand = input_hh + input_cts_ind

    e = round((output_demand - input_demand) / input_demand, 2) * 100

    print(f"electricity demand: {e} %")


def etrago_eGon2035_heat():
    """Execute basic sanity checks.

    Returns print statements as sanity checks for the heat sector in
    the eGon2035 scenario.

    Parameters
    ----------
    None

    Returns
    -------
    None
    """

    # Check input and output values for the carriers "other_non_renewable",
    # "other_renewable", "reservoir", "run_of_river" and "oil"

    scn = "eGon2035"

    # Section to check generator capacities
    print(f"Sanity checks for scenario {scn}")
    print(
        "For German heat demands the following deviations between the inputs"
        " and outputs can be observed:"
    )

    # Sanity checks for heat demand

    output_heat_demand = db.select_dataframe(
        """SELECT a.scn_name,
          (SUM(
          (SELECT SUM(p) FROM UNNEST(b.p_set) p))/1000000)::numeric as load_twh
            FROM grid.egon_etrago_load a
            JOIN grid.egon_etrago_load_timeseries b
            ON (a.load_id = b.load_id)
            JOIN grid.egon_etrago_bus c
            ON (a.bus=c.bus_id)
            AND b.scn_name = 'eGon2035'
            AND a.scn_name = 'eGon2035'
            AND c.scn_name= 'eGon2035'
            AND c.country='DE'
            AND a.carrier IN ('rural_heat', 'central_heat')
            GROUP BY (a.scn_name);
        """,
        warning=False,
    )["load_twh"].values[0]

    input_heat_demand = db.select_dataframe(
        """SELECT scenario, SUM(demand::numeric/1000000) as demand_mw_peta_heat
            FROM demand.egon_peta_heat
            WHERE scenario= 'eGon2035'
            GROUP BY (scenario);
        """,
        warning=False,
    )["demand_mw_peta_heat"].values[0]

    e_demand = (
        round((output_heat_demand - input_heat_demand) / input_heat_demand, 2)
        * 100
    )

    print(f"heat demand: {e_demand} %")

    # Sanity checks for heat supply

    print(
        "For German heat supplies the following deviations between the inputs "
        "and outputs can be observed:"
    )

    # Comparison for central heat pumps
    heat_pump_input = db.select_dataframe(
        """SELECT carrier, SUM(capacity::numeric) as Urban_central_heat_pump_mw
            FROM supply.egon_scenario_capacities
            WHERE carrier= 'urban_central_heat_pump'
            AND scenario_name IN ('eGon2035')
            GROUP BY (carrier);
        """,
        warning=False,
    )["urban_central_heat_pump_mw"].values[0]

    heat_pump_output = db.select_dataframe(
        """SELECT carrier, SUM(p_nom::numeric) as Central_heat_pump_mw
            FROM grid.egon_etrago_link
            WHERE carrier= 'central_heat_pump'
            AND scn_name IN ('eGon2035')
            GROUP BY (carrier);
    """,
        warning=False,
    )["central_heat_pump_mw"].values[0]

    e_heat_pump = (
        round((heat_pump_output - heat_pump_input) / heat_pump_output, 2) * 100
    )

    print(f"'central_heat_pump': {e_heat_pump} % ")

    # Comparison for residential heat pumps

    input_residential_heat_pump = db.select_dataframe(
        """SELECT carrier, SUM(capacity::numeric) as residential_heat_pump_mw
            FROM supply.egon_scenario_capacities
            WHERE carrier= 'residential_rural_heat_pump'
            AND scenario_name IN ('eGon2035')
            GROUP BY (carrier);
        """,
        warning=False,
    )["residential_heat_pump_mw"].values[0]

    output_residential_heat_pump = db.select_dataframe(
        """SELECT carrier, SUM(p_nom::numeric) as rural_heat_pump_mw
            FROM grid.egon_etrago_link
            WHERE carrier= 'rural_heat_pump'
            AND scn_name IN ('eGon2035')
            GROUP BY (carrier);
    """,
        warning=False,
    )["rural_heat_pump_mw"].values[0]

    e_residential_heat_pump = (
        round(
            (output_residential_heat_pump - input_residential_heat_pump)
            / input_residential_heat_pump,
            2,
        )
        * 100
    )
    print(f"'residential heat pumps': {e_residential_heat_pump} %")

    # Comparison for resistive heater
    resistive_heater_input = db.select_dataframe(
        """SELECT carrier,
         SUM(capacity::numeric) as Urban_central_resistive_heater_MW
            FROM supply.egon_scenario_capacities
            WHERE carrier= 'urban_central_resistive_heater'
            AND scenario_name IN ('eGon2035')
            GROUP BY (carrier);
        """,
        warning=False,
    )["urban_central_resistive_heater_mw"].values[0]

    resistive_heater_output = db.select_dataframe(
        """SELECT carrier, SUM(p_nom::numeric) as central_resistive_heater_MW
            FROM grid.egon_etrago_link
            WHERE carrier= 'central_resistive_heater'
            AND scn_name IN ('eGon2035')
            GROUP BY (carrier);
        """,
        warning=False,
    )["central_resistive_heater_mw"].values[0]

    e_resistive_heater = (
        round(
            (resistive_heater_output - resistive_heater_input)
            / resistive_heater_input,
            2,
        )
        * 100
    )

    print(f"'resistive heater': {e_resistive_heater} %")

    # Comparison for solar thermal collectors

    input_solar_thermal = db.select_dataframe(
        """SELECT carrier, SUM(capacity::numeric) as solar_thermal_collector_mw
            FROM supply.egon_scenario_capacities
            WHERE carrier= 'urban_central_solar_thermal_collector'
            AND scenario_name IN ('eGon2035')
            GROUP BY (carrier);
        """,
        warning=False,
    )["solar_thermal_collector_mw"].values[0]

    output_solar_thermal = db.select_dataframe(
        """SELECT carrier, SUM(p_nom::numeric) as solar_thermal_collector_mw
            FROM grid.egon_etrago_generator
            WHERE carrier= 'solar_thermal_collector'
            AND scn_name IN ('eGon2035')
            GROUP BY (carrier);
        """,
        warning=False,
    )["solar_thermal_collector_mw"].values[0]

    e_solar_thermal = (
        round(
            (output_solar_thermal - input_solar_thermal) / input_solar_thermal,
            2,
        )
        * 100
    )
    print(f"'solar thermal collector': {e_solar_thermal} %")

    # Comparison for geothermal

    input_geo_thermal = db.select_dataframe(
        """SELECT carrier,
         SUM(capacity::numeric) as Urban_central_geo_thermal_MW
            FROM supply.egon_scenario_capacities
            WHERE carrier= 'urban_central_geo_thermal'
            AND scenario_name IN ('eGon2035')
            GROUP BY (carrier);
        """,
        warning=False,
    )["urban_central_geo_thermal_mw"].values[0]

    output_geo_thermal = db.select_dataframe(
        """SELECT carrier, SUM(p_nom::numeric) as geo_thermal_MW
            FROM grid.egon_etrago_generator
            WHERE carrier= 'geo_thermal'
            AND scn_name IN ('eGon2035')
            GROUP BY (carrier);
    """,
        warning=False,
    )["geo_thermal_mw"].values[0]

    e_geo_thermal = (
        round((output_geo_thermal - input_geo_thermal) / input_geo_thermal, 2)
        * 100
    )
    print(f"'geothermal': {e_geo_thermal} %")


def residential_electricity_annual_sum(rtol=1e-5):
    """Sanity check for dataset electricity_demand_timeseries

    Aggregate the annual demand of all census cells at NUTS3 to compare
    with initial scaling parameters from DemandRegio.
    """

    df_nuts3_annual_sum = db.select_dataframe(
        sql="""
        SELECT dr.nuts3, dr.scenario, dr.demand_regio_sum, profiles.profile_sum
        FROM (
            SELECT scenario, SUM(demand) AS profile_sum, vg250_nuts3
            FROM demand.egon_demandregio_zensus_electricity AS egon,
             boundaries.egon_map_zensus_vg250 AS boundaries
            Where egon.zensus_population_id = boundaries.zensus_population_id
            AND sector = 'residential'
            GROUP BY vg250_nuts3, scenario
            ) AS profiles
        JOIN (
            SELECT nuts3, scenario, sum(demand) AS demand_regio_sum
            FROM demand.egon_demandregio_hh
            GROUP BY year, scenario, nuts3
              ) AS dr
        ON profiles.vg250_nuts3 = dr.nuts3 and profiles.scenario  = dr.scenario
        """
    )

    np.testing.assert_allclose(
        actual=df_nuts3_annual_sum["profile_sum"],
        desired=df_nuts3_annual_sum["demand_regio_sum"],
        rtol=rtol,
        verbose=False,
    )

    print(
        "Aggregated annual residential electricity demand"
        " matches with DemandRegio at NUTS-3."
    )


def residential_electricity_hh_refinement(rtol=1e-5):
    """Sanity check for dataset electricity_demand_timeseries

    Check sum of aggregated household types after refinement method
    was applied and compare it to the original census values."""

    df_refinement = db.select_dataframe(
        sql="""
        SELECT refined.nuts3, refined.characteristics_code,
                refined.sum_refined::int, census.sum_census::int
        FROM(
            SELECT nuts3, characteristics_code, SUM(hh_10types) as sum_refined
            FROM society.egon_destatis_zensus_household_per_ha_refined
            GROUP BY nuts3, characteristics_code)
            AS refined
        JOIN(
            SELECT t.nuts3, t.characteristics_code, sum(orig) as sum_census
            FROM(
                SELECT nuts3, cell_id, characteristics_code,
                        sum(DISTINCT(hh_5types))as orig
                FROM society.egon_destatis_zensus_household_per_ha_refined
                GROUP BY cell_id, characteristics_code, nuts3) AS t
            GROUP BY t.nuts3, t.characteristics_code    ) AS census
        ON refined.nuts3 = census.nuts3
        AND refined.characteristics_code = census.characteristics_code
    """
    )

    np.testing.assert_allclose(
        actual=df_refinement["sum_refined"],
        desired=df_refinement["sum_census"],
        rtol=rtol,
        verbose=False,
    )

    print("All Aggregated household types match at NUTS-3.")


1		"""
2		This module does sanity checks for both the eGon2035 and the eGon100RE scenario
3		separately where a percentage error is given to showcase difference in output
4		and input values. Please note that there are missing input technologies in the
5		supply tables.
6		Authors: @ALonso, @dana
7		"""
8
9		import numpy as np
10
11		from egon.data import db
12		from egon.data.datasets import Dataset
13
14
15		class SanityChecks(Dataset):
16		def __init__(self, dependencies):
17		super().__init__(
18		name="SanityChecks",
19		version="0.0.3",
20		dependencies=dependencies,
21		tasks={
22		etrago_eGon2035_electricity,
23		etrago_eGon2035_heat,
24		residential_electricity_annual_sum,
25		residential_electricity_hh_refinement,
26		},
27		)
28
29
30		def etrago_eGon2035_electricity():
31		"""Execute basic sanity checks.
32
33		Returns print statements as sanity checks for the electricity sector in
34		the eGon2035 scenario.
35
36		Parameters
37		----------
38		None
39
40		Returns
41		-------
42		None
43		"""
44
45		scn = "eGon2035"
46
47		# Section to check generator capacities
48		print(f"Sanity checks for scenario {scn}")
49		print(
50		"For German electricity generators the following deviations between "
51		"the inputs and outputs can be observed:"
52		)
53
54		carriers_electricity = [
55		"other_non_renewable",
56		"other_renewable",
57		"reservoir",
58		"run_of_river",
59		"oil",
60		"wind_onshore",
61		"wind_offshore",
62		"solar",
63		"solar_rooftop",
64		"biomass",
65		]
66
67		for carrier in carriers_electricity:
68
69		if carrier == "biomass":
70		sum_output = db.select_dataframe(
71		"""SELECT scn_name, SUM(p_nom::numeric) as output_capacity_mw
72		FROM grid.egon_etrago_generator
73		WHERE bus IN (
74		SELECT bus_id FROM grid.egon_etrago_bus
75		WHERE scn_name = 'eGon2035'
76		AND country = 'DE')
77		AND carrier IN ('biomass', 'industrial_biomass_CHP',
78		'central_biomass_CHP')
79		GROUP BY (scn_name);
80		""",
81		warning=False,
82		)
83
84		else:
85		sum_output = db.select_dataframe(
86		f"""SELECT scn_name,
87		SUM(p_nom::numeric) as output_capacity_mw
88		FROM grid.egon_etrago_generator
89		WHERE scn_name = '{scn}'
90		AND carrier IN ('{carrier}')
91		AND bus IN
92		(SELECT bus_id
93		FROM grid.egon_etrago_bus
94		WHERE scn_name = 'eGon2035'
95		AND country = 'DE')
96		GROUP BY (scn_name);
97		""",
98		warning=False,
99		)
100
101		sum_input = db.select_dataframe(
102		f"""SELECT carrier, SUM(capacity::numeric) as input_capacity_mw
103		FROM supply.egon_scenario_capacities
104		WHERE carrier= '{carrier}'
105		AND scenario_name ='{scn}'
106		GROUP BY (carrier);
107		""",
108		warning=False,
109		)
110
111	View Code Duplication	if (
		0 ignored issues – show Duplication introduced 2022-06-03 07:55 UTC by Report Bug Copy Issue Report This code seems to be duplicated in your project. Loading history...
112		sum_output.output_capacity_mw.sum() == 0
113		and sum_input.input_capacity_mw.sum() == 0
114		):
115		print(
116		f"No capacity for carrier '{carrier}' needed to be"
117		f" distributed. Everything is fine"
118		)
119
120		elif (
121		sum_input.input_capacity_mw.sum() > 0
122		and sum_output.output_capacity_mw.sum() == 0
123		):
124		print(
125		f"Error: Capacity for carrier '{carrier}' was not distributed "
126		f"at all!"
127		)
128
129		elif (
130		sum_output.output_capacity_mw.sum() > 0
131		and sum_input.input_capacity_mw.sum() == 0
132		):
133		print(
134		f"Error: Eventhough no input capacity was provided for carrier"
135		f"'{carrier}' a capacity got distributed!"
136		)
137
138		else:
139		sum_input["error"] = (
140		(sum_output.output_capacity_mw - sum_input.input_capacity_mw)
141		/ sum_input.input_capacity_mw
142		) * 100
143		g = sum_input["error"].values[0]
144
145		print(f"{carrier}: " + str(round(g, 2)) + " %")
146
147		# Section to check storage units
148
149		print(f"Sanity checks for scenario {scn}")
150		print(
151		"For German electrical storage units the following deviations between"
152		"the inputs and outputs can be observed:"
153		)
154
155		carriers_electricity = ["pumped_hydro"]
156
157		for carrier in carriers_electricity:
158
159		sum_output = db.select_dataframe(
160		f"""SELECT scn_name, SUM(p_nom::numeric) as output_capacity_mw
161		FROM grid.egon_etrago_storage
162		WHERE scn_name = '{scn}'
163		AND carrier IN ('{carrier}')
164		AND bus IN
165		(SELECT bus_id
166		FROM grid.egon_etrago_bus
167		WHERE scn_name = 'eGon2035'
168		AND country = 'DE')
169		GROUP BY (scn_name);
170		""",
171		warning=False,
172		)
173
174		sum_input = db.select_dataframe(
175		f"""SELECT carrier, SUM(capacity::numeric) as input_capacity_mw
176		FROM supply.egon_scenario_capacities
177		WHERE carrier= '{carrier}'
178		AND scenario_name ='{scn}'
179		GROUP BY (carrier);
180		""",
181		warning=False,
182		)
183
184	View Code Duplication	if (
		0 ignored issues – show Duplication introduced 2022-06-03 07:55 UTC by Report Bug Copy Issue Report This code seems to be duplicated in your project. Loading history...
185		sum_output.output_capacity_mw.sum() == 0
186		and sum_input.input_capacity_mw.sum() == 0
187		):
188		print(
189		f"No capacity for carrier '{carrier}' needed to be "
190		f"distributed. Everything is fine"
191		)
192
193		elif (
194		sum_input.input_capacity_mw.sum() > 0
195		and sum_output.output_capacity_mw.sum() == 0
196		):
197		print(
198		f"Error: Capacity for carrier '{carrier}' was not distributed"
199		f" at all!"
200		)
201
202		elif (
203		sum_output.output_capacity_mw.sum() > 0
204		and sum_input.input_capacity_mw.sum() == 0
205		):
206		print(
207		f"Error: Eventhough no input capacity was provided for carrier"
208		f" '{carrier}' a capacity got distributed!"
209		)
210
211		else:
212		sum_input["error"] = (
213		(sum_output.output_capacity_mw - sum_input.input_capacity_mw)
214		/ sum_input.input_capacity_mw
215		) * 100
216		g = sum_input["error"].values[0]
217
218		print(f"{carrier}: " + str(round(g, 2)) + " %")
219
220		# Section to check loads
221
222		print(
223		"For German electricity loads the following deviations between the"
224		" input and output can be observed:"
225		)
226
227		output_demand = db.select_dataframe(
228		"""SELECT a.scn_name, a.carrier, SUM((SELECT SUM(p)
229		FROM UNNEST(b.p_set) p))/1000000::numeric as load_twh
230		FROM grid.egon_etrago_load a
231		JOIN grid.egon_etrago_load_timeseries b
232		ON (a.load_id = b.load_id)
233		JOIN grid.egon_etrago_bus c
234		ON (a.bus=c.bus_id)
235		AND b.scn_name = 'eGon2035'
236		AND a.scn_name = 'eGon2035'
237		AND a.carrier = 'AC'
238		AND c.scn_name= 'eGon2035'
239		AND c.country='DE'
240		GROUP BY (a.scn_name, a.carrier);
241
242		""",
243		warning=False,
244		)["load_twh"].values[0]
245
246		input_cts_ind = db.select_dataframe(
247		"""SELECT scenario,
248		SUM(demand::numeric/1000000) as demand_mw_regio_cts_ind
249		FROM demand.egon_demandregio_cts_ind
250		WHERE scenario= 'eGon2035'
251		AND year IN ('2035')
252		GROUP BY (scenario);
253
254		""",
255		warning=False,
256		)["demand_mw_regio_cts_ind"].values[0]
257
258		input_hh = db.select_dataframe(
259		"""SELECT scenario, SUM(demand::numeric/1000000) as demand_mw_regio_hh
260		FROM demand.egon_demandregio_hh
261		WHERE scenario= 'eGon2035'
262		AND year IN ('2035')
263		GROUP BY (scenario);
264		""",
265		warning=False,
266		)["demand_mw_regio_hh"].values[0]
267
268		input_demand = input_hh + input_cts_ind
269
270		e = round((output_demand - input_demand) / input_demand, 2) * 100
271
272		print(f"electricity demand: {e} %")
273
274
275		def etrago_eGon2035_heat():
276		"""Execute basic sanity checks.
277
278		Returns print statements as sanity checks for the heat sector in
279		the eGon2035 scenario.
280
281		Parameters
282		----------
283		None
284
285		Returns
286		-------
287		None
288		"""
289
290		# Check input and output values for the carriers "other_non_renewable",
291		# "other_renewable", "reservoir", "run_of_river" and "oil"
292
293		scn = "eGon2035"
294
295		# Section to check generator capacities
296		print(f"Sanity checks for scenario {scn}")
297		print(
298		"For German heat demands the following deviations between the inputs"
299		" and outputs can be observed:"
300		)
301
302		# Sanity checks for heat demand
303
304		output_heat_demand = db.select_dataframe(
305		"""SELECT a.scn_name,
306		(SUM(
307		(SELECT SUM(p) FROM UNNEST(b.p_set) p))/1000000)::numeric as load_twh
308		FROM grid.egon_etrago_load a
309		JOIN grid.egon_etrago_load_timeseries b
310		ON (a.load_id = b.load_id)
311		JOIN grid.egon_etrago_bus c
312		ON (a.bus=c.bus_id)
313		AND b.scn_name = 'eGon2035'
314		AND a.scn_name = 'eGon2035'
315		AND c.scn_name= 'eGon2035'
316		AND c.country='DE'
317		AND a.carrier IN ('rural_heat', 'central_heat')
318		GROUP BY (a.scn_name);
319		""",
320		warning=False,
321		)["load_twh"].values[0]
322
323		input_heat_demand = db.select_dataframe(
324		"""SELECT scenario, SUM(demand::numeric/1000000) as demand_mw_peta_heat
325		FROM demand.egon_peta_heat
326		WHERE scenario= 'eGon2035'
327		GROUP BY (scenario);
328		""",
329		warning=False,
330		)["demand_mw_peta_heat"].values[0]
331
332		e_demand = (
333		round((output_heat_demand - input_heat_demand) / input_heat_demand, 2)
334		* 100
335		)
336
337		print(f"heat demand: {e_demand} %")
338
339		# Sanity checks for heat supply
340
341		print(
342		"For German heat supplies the following deviations between the inputs "
343		"and outputs can be observed:"
344		)
345
346		# Comparison for central heat pumps
347		heat_pump_input = db.select_dataframe(
348		"""SELECT carrier, SUM(capacity::numeric) as Urban_central_heat_pump_mw
349		FROM supply.egon_scenario_capacities
350		WHERE carrier= 'urban_central_heat_pump'
351		AND scenario_name IN ('eGon2035')
352		GROUP BY (carrier);
353		""",
354		warning=False,
355		)["urban_central_heat_pump_mw"].values[0]
356
357		heat_pump_output = db.select_dataframe(
358		"""SELECT carrier, SUM(p_nom::numeric) as Central_heat_pump_mw
359		FROM grid.egon_etrago_link
360		WHERE carrier= 'central_heat_pump'
361		AND scn_name IN ('eGon2035')
362		GROUP BY (carrier);
363		""",
364		warning=False,
365		)["central_heat_pump_mw"].values[0]
366
367		e_heat_pump = (
368		round((heat_pump_output - heat_pump_input) / heat_pump_output, 2) * 100
369		)
370
371		print(f"'central_heat_pump': {e_heat_pump} % ")
372
373		# Comparison for residential heat pumps
374
375		input_residential_heat_pump = db.select_dataframe(
376		"""SELECT carrier, SUM(capacity::numeric) as residential_heat_pump_mw
377		FROM supply.egon_scenario_capacities
378		WHERE carrier= 'residential_rural_heat_pump'
379		AND scenario_name IN ('eGon2035')
380		GROUP BY (carrier);
381		""",
382		warning=False,
383		)["residential_heat_pump_mw"].values[0]
384
385		output_residential_heat_pump = db.select_dataframe(
386		"""SELECT carrier, SUM(p_nom::numeric) as rural_heat_pump_mw
387		FROM grid.egon_etrago_link
388		WHERE carrier= 'rural_heat_pump'
389		AND scn_name IN ('eGon2035')
390		GROUP BY (carrier);
391		""",
392		warning=False,
393		)["rural_heat_pump_mw"].values[0]
394
395		e_residential_heat_pump = (
396		round(
397		(output_residential_heat_pump - input_residential_heat_pump)
398		/ input_residential_heat_pump,
399		2,
400		)
401		* 100
402		)
403		print(f"'residential heat pumps': {e_residential_heat_pump} %")
404
405		# Comparison for resistive heater
406		resistive_heater_input = db.select_dataframe(
407		"""SELECT carrier,
408		SUM(capacity::numeric) as Urban_central_resistive_heater_MW
409		FROM supply.egon_scenario_capacities
410		WHERE carrier= 'urban_central_resistive_heater'
411		AND scenario_name IN ('eGon2035')
412		GROUP BY (carrier);
413		""",
414		warning=False,
415		)["urban_central_resistive_heater_mw"].values[0]
416
417		resistive_heater_output = db.select_dataframe(
418		"""SELECT carrier, SUM(p_nom::numeric) as central_resistive_heater_MW
419		FROM grid.egon_etrago_link
420		WHERE carrier= 'central_resistive_heater'
421		AND scn_name IN ('eGon2035')
422		GROUP BY (carrier);
423		""",
424		warning=False,
425		)["central_resistive_heater_mw"].values[0]
426
427		e_resistive_heater = (
428		round(
429		(resistive_heater_output - resistive_heater_input)
430		/ resistive_heater_input,
431		2,
432		)
433		* 100
434		)
435
436		print(f"'resistive heater': {e_resistive_heater} %")
437
438		# Comparison for solar thermal collectors
439
440		input_solar_thermal = db.select_dataframe(
441		"""SELECT carrier, SUM(capacity::numeric) as solar_thermal_collector_mw
442		FROM supply.egon_scenario_capacities
443		WHERE carrier= 'urban_central_solar_thermal_collector'
444		AND scenario_name IN ('eGon2035')
445		GROUP BY (carrier);
446		""",
447		warning=False,
448		)["solar_thermal_collector_mw"].values[0]
449
450		output_solar_thermal = db.select_dataframe(
451		"""SELECT carrier, SUM(p_nom::numeric) as solar_thermal_collector_mw
452		FROM grid.egon_etrago_generator
453		WHERE carrier= 'solar_thermal_collector'
454		AND scn_name IN ('eGon2035')
455		GROUP BY (carrier);
456		""",
457		warning=False,
458		)["solar_thermal_collector_mw"].values[0]
459
460		e_solar_thermal = (
461		round(
462		(output_solar_thermal - input_solar_thermal) / input_solar_thermal,
463		2,
464		)
465		* 100
466		)
467		print(f"'solar thermal collector': {e_solar_thermal} %")
468
469		# Comparison for geothermal
470
471		input_geo_thermal = db.select_dataframe(
472		"""SELECT carrier,
473		SUM(capacity::numeric) as Urban_central_geo_thermal_MW
474		FROM supply.egon_scenario_capacities
475		WHERE carrier= 'urban_central_geo_thermal'
476		AND scenario_name IN ('eGon2035')
477		GROUP BY (carrier);
478		""",
479		warning=False,
480		)["urban_central_geo_thermal_mw"].values[0]
481
482		output_geo_thermal = db.select_dataframe(
483		"""SELECT carrier, SUM(p_nom::numeric) as geo_thermal_MW
484		FROM grid.egon_etrago_generator
485		WHERE carrier= 'geo_thermal'
486		AND scn_name IN ('eGon2035')
487		GROUP BY (carrier);
488		""",
489		warning=False,
490		)["geo_thermal_mw"].values[0]
491
492		e_geo_thermal = (
493		round((output_geo_thermal - input_geo_thermal) / input_geo_thermal, 2)
494		* 100
495		)
496		print(f"'geothermal': {e_geo_thermal} %")
497
498
499		def residential_electricity_annual_sum(rtol=1e-5):
500		"""Sanity check for dataset electricity_demand_timeseries
501
502		Aggregate the annual demand of all census cells at NUTS3 to compare
503		with initial scaling parameters from DemandRegio.
504		"""
505
506		df_nuts3_annual_sum = db.select_dataframe(
507		sql="""
508		SELECT dr.nuts3, dr.scenario, dr.demand_regio_sum, profiles.profile_sum
509		FROM (
510		SELECT scenario, SUM(demand) AS profile_sum, vg250_nuts3
511		FROM demand.egon_demandregio_zensus_electricity AS egon,
512		boundaries.egon_map_zensus_vg250 AS boundaries
513		Where egon.zensus_population_id = boundaries.zensus_population_id
514		AND sector = 'residential'
515		GROUP BY vg250_nuts3, scenario
516		) AS profiles
517		JOIN (
518		SELECT nuts3, scenario, sum(demand) AS demand_regio_sum
519		FROM demand.egon_demandregio_hh
520		GROUP BY year, scenario, nuts3
521		) AS dr
522		ON profiles.vg250_nuts3 = dr.nuts3 and profiles.scenario = dr.scenario
523		"""
524		)
525
526		np.testing.assert_allclose(
527		actual=df_nuts3_annual_sum["profile_sum"],
528		desired=df_nuts3_annual_sum["demand_regio_sum"],
529		rtol=rtol,
530		verbose=False,
531		)
532
533		print(
534		"Aggregated annual residential electricity demand"
535		" matches with DemandRegio at NUTS-3."
536		)
537
538
539		def residential_electricity_hh_refinement(rtol=1e-5):
540		"""Sanity check for dataset electricity_demand_timeseries
541
542		Check sum of aggregated household types after refinement method
543		was applied and compare it to the original census values."""
544
545		df_refinement = db.select_dataframe(
546		sql="""
547		SELECT refined.nuts3, refined.characteristics_code,
548		refined.sum_refined::int, census.sum_census::int
549		FROM(
550		SELECT nuts3, characteristics_code, SUM(hh_10types) as sum_refined
551		FROM society.egon_destatis_zensus_household_per_ha_refined
552		GROUP BY nuts3, characteristics_code)
553		AS refined
554		JOIN(
555		SELECT t.nuts3, t.characteristics_code, sum(orig) as sum_census
556		FROM(
557		SELECT nuts3, cell_id, characteristics_code,
558		sum(DISTINCT(hh_5types))as orig
559		FROM society.egon_destatis_zensus_household_per_ha_refined
560		GROUP BY cell_id, characteristics_code, nuts3) AS t
561		GROUP BY t.nuts3, t.characteristics_code ) AS census
562		ON refined.nuts3 = census.nuts3
563		AND refined.characteristics_code = census.characteristics_code
564		"""
565		)
566
567		np.testing.assert_allclose(
568		actual=df_refinement["sum_refined"],
569		desired=df_refinement["sum_census"],
570		rtol=rtol,
571		verbose=False,
572		)
573
574		print("All Aggregated household types match at NUTS-3.")
575

openego / eGon-data

Pull Request — dev (#915)

sanitycheck_eGon2035_electricity() F

Complexity

Size

Duplication

Importance

How to fix Long Method Complexity

Long Method

Complexity

Duplication Side-by-Side

Filter issues like